aboutsummaryrefslogtreecommitdiffstats
path: root/drivers/dma/ioat (follow)
AgeCommit message (Collapse)AuthorFilesLines
2019-02-25dmaengine: ioatdma: support latency tolerance report (LTR) for v3.4Dave Jiang2-0/+41
IOATDMA 3.4 supports PCIe LTR mechanism. The registers are non-standard PCIe LTR support. This needs to be setup in order to not suffer performance impact and provide proper power management. The channel is set to active when it is allocated, and to passive when it's freed. Signed-off-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Vinod Koul <vkoul@kernel.org>
2019-02-25dmaengine: ioatdma: add descriptor pre-fetch support for v3.4Dave Jiang3-2/+28
Adding support for new feature on ioatdma 3.4 hardware that provides descriptor pre-fetching in order to reduce small DMA latencies. Signed-off-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Vinod Koul <vkoul@kernel.org>
2019-02-25dmaengine: ioatdma: disable DCA enabling on IOATDMA v3.4Dave Jiang2-0/+3
IOATDMA v3.4 does not support DCA. Disable Signed-off-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Vinod Koul <vkoul@kernel.org>
2019-02-25dmaengine: ioatdma: Add Snow Ridge ioatdma device idDave Jiang3-1/+6
Add Snowridge Xeon-D ioatdma PCI device id. Also applies for Icelake SP Xeon. This introduces ioatdma v3.4 platform. Also bumping driver version to 5.0 since we are adding additional code for 3.4 support. Signed-off-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Vinod Koul <vkoul@kernel.org>
2018-10-25Merge tag 'pci-v4.20-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pciLinus Torvalds1-7/+0
Pull PCI updates from Bjorn Helgaas: - Fix ASPM link_state teardown on removal (Lukas Wunner) - Fix misleading _OSC ASPM message (Sinan Kaya) - Make _OSC optional for PCI (Sinan Kaya) - Don't initialize ASPM link state when ACPI_FADT_NO_ASPM is set (Patrick Talbert) - Remove x86 and arm64 node-local allocation for host bridge structures (Punit Agrawal) - Pay attention to device-specific _PXM node values (Jonathan Cameron) - Support new Immediate Readiness bit (Felipe Balbi) - Differentiate between pciehp surprise and safe removal (Lukas Wunner) - Remove unnecessary pciehp includes (Lukas Wunner) - Drop pciehp hotplug_slot_ops wrappers (Lukas Wunner) - Tolerate PCIe Slot Presence Detect being hardwired to zero to workaround broken hardware, e.g., the Wilocity switch/wireless device (Lukas Wunner) - Unify pciehp controller & slot structs (Lukas Wunner) - Constify hotplug_slot_ops (Lukas Wunner) - Drop hotplug_slot_info (Lukas Wunner) - Embed hotplug_slot struct into users instead of allocating it separately (Lukas Wunner) - Initialize PCIe port service drivers directly instead of relying on initcall ordering (Keith Busch) - Restore PCI config state after a slot reset (Keith Busch) - Save/restore DPC config state along with other PCI config state (Keith Busch) - Reference count devices during AER handling to avoid race issue with concurrent hot removal (Keith Busch) - If an Upstream Port reports ERR_FATAL, don't try to read the Port's config space because it is probably unreachable (Keith Busch) - During error handling, use slot-specific reset instead of secondary bus reset to avoid link up/down issues on hotplug ports (Keith Busch) - Restore previous AER/DPC handling that does not remove and re-enumerate devices on ERR_FATAL (Keith Busch) - Notify all drivers that may be affected by error recovery resets (Keith Busch) - Always generate error recovery uevents, even if a driver doesn't have error callbacks (Keith Busch) - Make PCIe link active reporting detection generic (Keith Busch) - Support D3cold in PCIe hierarchies during system sleep and runtime, including hotplug and Thunderbolt ports (Mika Westerberg) - Handle hpmemsize/hpiosize kernel parameters uniformly, whether slots are empty or occupied (Jon Derrick) - Remove duplicated include from pci/pcie/err.c and unused variable from cpqphp (YueHaibing) - Remove driver pci_cleanup_aer_uncorrect_error_status() calls (Oza Pawandeep) - Uninline PCI bus accessors for better ftracing (Keith Busch) - Remove unused AER Root Port .error_resume method (Keith Busch) - Use kfifo in AER instead of a local version (Keith Busch) - Use threaded IRQ in AER bottom half (Keith Busch) - Use managed resources in AER core (Keith Busch) - Reuse pcie_port_find_device() for AER injection (Keith Busch) - Abstract AER interrupt handling to disconnect error injection (Keith Busch) - Refactor AER injection callbacks to simplify future improvments (Keith Busch) - Remove unused Netronome NFP32xx Device IDs (Jakub Kicinski) - Use bitmap_zalloc() for dma_alias_mask (Andy Shevchenko) - Add switch fall-through annotations (Gustavo A. R. Silva) - Remove unused Switchtec quirk variable (Joshua Abraham) - Fix pci.c kernel-doc warning (Randy Dunlap) - Remove trivial PCI wrappers for DMA APIs (Christoph Hellwig) - Add Intel GPU device IDs to spurious interrupt quirk (Bin Meng) - Run Switchtec DMA aliasing quirk only on NTB endpoints to avoid useless dmesg errors (Logan Gunthorpe) - Update Switchtec NTB documentation (Wesley Yung) - Remove redundant "default n" from Kconfig (Bartlomiej Zolnierkiewicz) - Avoid panic when drivers enable MSI/MSI-X twice (Tonghao Zhang) - Add PCI support for peer-to-peer DMA (Logan Gunthorpe) - Add sysfs group for PCI peer-to-peer memory statistics (Logan Gunthorpe) - Add PCI peer-to-peer DMA scatterlist mapping interface (Logan Gunthorpe) - Add PCI configfs/sysfs helpers for use by peer-to-peer users (Logan Gunthorpe) - Add PCI peer-to-peer DMA driver writer's documentation (Logan Gunthorpe) - Add block layer flag to indicate driver support for PCI peer-to-peer DMA (Logan Gunthorpe) - Map Infiniband scatterlists for peer-to-peer DMA if they contain P2P memory (Logan Gunthorpe) - Register nvme-pci CMB buffer as PCI peer-to-peer memory (Logan Gunthorpe) - Add nvme-pci support for PCI peer-to-peer memory in requests (Logan Gunthorpe) - Use PCI peer-to-peer memory in nvme (Stephen Bates, Steve Wise, Christoph Hellwig, Logan Gunthorpe) - Cache VF config space size to optimize enumeration of many VFs (KarimAllah Ahmed) - Remove unnecessary <linux/pci-ats.h> include (Bjorn Helgaas) - Fix VMD AERSID quirk Device ID matching (Jon Derrick) - Fix Cadence PHY handling during probe (Alan Douglas) - Signal Cadence Endpoint interrupts via AXI region 0 instead of last region (Alan Douglas) - Write Cadence Endpoint MSI interrupts with 32 bits of data (Alan Douglas) - Remove redundant controller tests for "device_type == pci" (Rob Herring) - Document R-Car E3 (R8A77990) bindings (Tho Vu) - Add device tree support for R-Car r8a7744 (Biju Das) - Drop unused mvebu PCIe capability code (Thomas Petazzoni) - Add shared PCI bridge emulation code (Thomas Petazzoni) - Convert mvebu to use shared PCI bridge emulation (Thomas Petazzoni) - Add aardvark Root Port emulation (Thomas Petazzoni) - Support 100MHz/200MHz refclocks for i.MX6 (Lucas Stach) - Add initial power management for i.MX7 (Leonard Crestez) - Add PME_Turn_Off support for i.MX7 (Leonard Crestez) - Fix qcom runtime power management error handling (Bjorn Andersson) - Update TI dra7xx unaligned access errata workaround for host mode as well as endpoint mode (Vignesh R) - Fix kirin section mismatch warning (Nathan Chancellor) - Remove iproc PAXC slot check to allow VF support (Jitendra Bhivare) - Quirk Keystone K2G to limit MRRS to 256 (Kishon Vijay Abraham I) - Update Keystone to use MRRS quirk for host bridge instead of open coding (Kishon Vijay Abraham I) - Refactor Keystone link establishment (Kishon Vijay Abraham I) - Simplify and speed up Keystone link training (Kishon Vijay Abraham I) - Remove unused Keystone host_init argument (Kishon Vijay Abraham I) - Merge Keystone driver files into one (Kishon Vijay Abraham I) - Remove redundant Keystone platform_set_drvdata() (Kishon Vijay Abraham I) - Rename Keystone functions for uniformity (Kishon Vijay Abraham I) - Add Keystone device control module DT binding (Kishon Vijay Abraham I) - Use SYSCON API to get Keystone control module device IDs (Kishon Vijay Abraham I) - Clean up Keystone PHY handling (Kishon Vijay Abraham I) - Use runtime PM APIs to enable Keystone clock (Kishon Vijay Abraham I) - Clean up Keystone config space access checks (Kishon Vijay Abraham I) - Get Keystone outbound window count from DT (Kishon Vijay Abraham I) - Clean up Keystone outbound window configuration (Kishon Vijay Abraham I) - Clean up Keystone DBI setup (Kishon Vijay Abraham I) - Clean up Keystone ks_pcie_link_up() (Kishon Vijay Abraham I) - Fix Keystone IRQ status checking (Kishon Vijay Abraham I) - Add debug messages for all Keystone errors (Kishon Vijay Abraham I) - Clean up Keystone includes and macros (Kishon Vijay Abraham I) - Fix Mediatek unchecked return value from devm_pci_remap_iospace() (Gustavo A. R. Silva) - Fix Mediatek endpoint/port matching logic (Honghui Zhang) - Change Mediatek Root Port Class Code to PCI_CLASS_BRIDGE_PCI (Honghui Zhang) - Remove redundant Mediatek PM domain check (Honghui Zhang) - Convert Mediatek to pci_host_probe() (Honghui Zhang) - Fix Mediatek MSI enablement (Honghui Zhang) - Add Mediatek system PM support for MT2712 and MT7622 (Honghui Zhang) - Add Mediatek loadable module support (Honghui Zhang) - Detach VMD resources after stopping root bus to prevent orphan resources (Jon Derrick) - Convert pcitest build process to that used by other tools (iio, perf, etc) (Gustavo Pimentel) * tag 'pci-v4.20-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci: (140 commits) PCI/AER: Refactor error injection fallbacks PCI/AER: Abstract AER interrupt handling PCI/AER: Reuse existing pcie_port_find_device() interface PCI/AER: Use managed resource allocations PCI: pcie: Remove redundant 'default n' from Kconfig PCI: aardvark: Implement emulated root PCI bridge config space PCI: mvebu: Convert to PCI emulated bridge config space PCI: mvebu: Drop unused PCI express capability code PCI: Introduce PCI bridge emulated config space common logic PCI: vmd: Detach resources after stopping root bus nvmet: Optionally use PCI P2P memory nvmet: Introduce helper functions to allocate and free request SGLs nvme-pci: Add support for P2P memory in requests nvme-pci: Use PCI p2pmem subsystem to manage the CMB IB/core: Ensure we map P2P memory correctly in rdma_rw_ctx_[init|destroy]() block: Add PCI P2P flag for request queue PCI/P2PDMA: Add P2P DMA driver writer's documentation docs-rst: Add a new directory for PCI documentation PCI/P2PDMA: Introduce configfs/sysfs enable attribute helpers PCI/P2PDMA: Add PCI p2pmem DMA mappings to adjust the bus offset ...
2018-10-05dmaengine: ioat: fix prototype of ioat_enumerate_channelsRami Rosen1-4/+3
Signed-off-by: Rami Rosen <ramirose@gmail.com> Signed-off-by: Vinod Koul <vkoul@kernel.org>
2018-10-02PCI/AER: Remove pci_cleanup_aer_uncorrect_error_status() callsOza Pawandeep1-7/+0
After bfcb79fca19d ("PCI/ERR: Run error recovery callbacks for all affected devices"), AER errors are always cleared by the PCI core and drivers don't need to do it themselves. Remove calls to pci_cleanup_aer_uncorrect_error_status() from device driver error recovery functions. Signed-off-by: Oza Pawandeep <poza@codeaurora.org> [bhelgaas: changelog, remove PCI core changes, remove unused variables] Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2018-09-18driver/dma/ioat: Call del_timer_sync() without holding prep_lockWaiman Long1-1/+8
The following lockdep splat was observed: [ 1222.241750] ====================================================== [ 1222.271301] WARNING: possible circular locking dependency detected [ 1222.301060] 4.16.0-10.el8+5.x86_64+debug #1 Not tainted [ 1222.326659] ------------------------------------------------------ [ 1222.356565] systemd-shutdow/1 is trying to acquire lock: [ 1222.382660] ((&ioat_chan->timer)){+.-.}, at: [<00000000f71e1a28>] del_timer_sync+0x5/0xf0 [ 1222.422928] [ 1222.422928] but task is already holding lock: [ 1222.451743] (&(&ioat_chan->prep_lock)->rlock){+.-.}, at: [<000000008ea98b12>] ioat_shutdown+0x86/0x100 [ioatdma] : [ 1223.524987] Chain exists of: [ 1223.524987] (&ioat_chan->timer) --> &(&ioat_chan->cleanup_lock)->rlock --> &(&ioat_chan->prep_lock)->rlock [ 1223.524987] [ 1223.594082] Possible unsafe locking scenario: [ 1223.594082] [ 1223.622630] CPU0 CPU1 [ 1223.645080] ---- ---- [ 1223.667404] lock(&(&ioat_chan->prep_lock)->rlock); [ 1223.691535] lock(&(&ioat_chan->cleanup_lock)->rlock); [ 1223.728657] lock(&(&ioat_chan->prep_lock)->rlock); [ 1223.765122] lock((&ioat_chan->timer)); [ 1223.784095] [ 1223.784095] *** DEADLOCK *** [ 1223.784095] [ 1223.813492] 4 locks held by systemd-shutdow/1: [ 1223.834677] #0: (reboot_mutex){+.+.}, at: [<0000000056d33456>] SYSC_reboot+0x10f/0x300 [ 1223.873310] #1: (&dev->mutex){....}, at: [<00000000258dfdd7>] device_shutdown+0x1c8/0x660 [ 1223.913604] #2: (&dev->mutex){....}, at: [<0000000068331147>] device_shutdown+0x1d6/0x660 [ 1223.954000] #3: (&(&ioat_chan->prep_lock)->rlock){+.-.}, at: [<000000008ea98b12>] ioat_shutdown+0x86/0x100 [ioatdma] In the ioat_shutdown() function: spin_lock_bh(&ioat_chan->prep_lock); set_bit(IOAT_CHAN_DOWN, &ioat_chan->state); del_timer_sync(&ioat_chan->timer); spin_unlock_bh(&ioat_chan->prep_lock); According to the synchronization rule for the del_timer_sync() function, the caller must not hold locks which would prevent completion of the timer's handler. The timer structure has its own lock that manages its synchronization. Setting the IOAT_CHAN_DOWN bit should prevent other CPUs from trying to use that device anyway, there is probably no need to call del_timer_sync() while holding the prep_lock. So the del_timer_sync() call is now moved outside of the prep_lock critical section to prevent the circular lock dependency. Signed-off-by: Waiman Long <longman@redhat.com> Reviewed-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Vinod Koul <vkoul@kernel.org>
2018-06-19dmaengine: ioatdma: set the completion address register after channel resetDave Jiang1-0/+6
It seems that starting with Skylake Xeon, channel reset clears the completion address register. Make sure the completion address register is set again after reset. Signed-off-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Vinod Koul <vkoul@kernel.org>
2018-06-12treewide: kzalloc() -> kcalloc()Kees Cook1-2/+2
The kzalloc() function has a 2-factor argument form, kcalloc(). This patch replaces cases of: kzalloc(a * b, gfp) with: kcalloc(a * b, gfp) as well as handling cases of: kzalloc(a * b * c, gfp) with: kzalloc(array3_size(a, b, c), gfp) as it's slightly less ugly than: kzalloc_array(array_size(a, b), c, gfp) This does, however, attempt to ignore constant size factors like: kzalloc(4 * 1024, gfp) though any constants defined via macros get caught up in the conversion. Any factors with a sizeof() of "unsigned char", "char", and "u8" were dropped, since they're redundant. The Coccinelle script used for this was: // Fix redundant parens around sizeof(). @@ type TYPE; expression THING, E; @@ ( kzalloc( - (sizeof(TYPE)) * E + sizeof(TYPE) * E , ...) | kzalloc( - (sizeof(THING)) * E + sizeof(THING) * E , ...) ) // Drop single-byte sizes and redundant parens. @@ expression COUNT; typedef u8; typedef __u8; @@ ( kzalloc( - sizeof(u8) * (COUNT) + COUNT , ...) | kzalloc( - sizeof(__u8) * (COUNT) + COUNT , ...) | kzalloc( - sizeof(char) * (COUNT) + COUNT , ...) | kzalloc( - sizeof(unsigned char) * (COUNT) + COUNT , ...) | kzalloc( - sizeof(u8) * COUNT + COUNT , ...) | kzalloc( - sizeof(__u8) * COUNT + COUNT , ...) | kzalloc( - sizeof(char) * COUNT + COUNT , ...) | kzalloc( - sizeof(unsigned char) * COUNT + COUNT , ...) ) // 2-factor product with sizeof(type/expression) and identifier or constant. @@ type TYPE; expression THING; identifier COUNT_ID; constant COUNT_CONST; @@ ( - kzalloc + kcalloc ( - sizeof(TYPE) * (COUNT_ID) + COUNT_ID, sizeof(TYPE) , ...) | - kzalloc + kcalloc ( - sizeof(TYPE) * COUNT_ID + COUNT_ID, sizeof(TYPE) , ...) | - kzalloc + kcalloc ( - sizeof(TYPE) * (COUNT_CONST) + COUNT_CONST, sizeof(TYPE) , ...) | - kzalloc + kcalloc ( - sizeof(TYPE) * COUNT_CONST + COUNT_CONST, sizeof(TYPE) , ...) | - kzalloc + kcalloc ( - sizeof(THING) * (COUNT_ID) + COUNT_ID, sizeof(THING) , ...) | - kzalloc + kcalloc ( - sizeof(THING) * COUNT_ID + COUNT_ID, sizeof(THING) , ...) | - kzalloc + kcalloc ( - sizeof(THING) * (COUNT_CONST) + COUNT_CONST, sizeof(THING) , ...) | - kzalloc + kcalloc ( - sizeof(THING) * COUNT_CONST + COUNT_CONST, sizeof(THING) , ...) ) // 2-factor product, only identifiers. @@ identifier SIZE, COUNT; @@ - kzalloc + kcalloc ( - SIZE * COUNT + COUNT, SIZE , ...) // 3-factor product with 1 sizeof(type) or sizeof(expression), with // redundant parens removed. @@ expression THING; identifier STRIDE, COUNT; type TYPE; @@ ( kzalloc( - sizeof(TYPE) * (COUNT) * (STRIDE) + array3_size(COUNT, STRIDE, sizeof(TYPE)) , ...) | kzalloc( - sizeof(TYPE) * (COUNT) * STRIDE + array3_size(COUNT, STRIDE, sizeof(TYPE)) , ...) | kzalloc( - sizeof(TYPE) * COUNT * (STRIDE) + array3_size(COUNT, STRIDE, sizeof(TYPE)) , ...) | kzalloc( - sizeof(TYPE) * COUNT * STRIDE + array3_size(COUNT, STRIDE, sizeof(TYPE)) , ...) | kzalloc( - sizeof(THING) * (COUNT) * (STRIDE) + array3_size(COUNT, STRIDE, sizeof(THING)) , ...) | kzalloc( - sizeof(THING) * (COUNT) * STRIDE + array3_size(COUNT, STRIDE, sizeof(THING)) , ...) | kzalloc( - sizeof(THING) * COUNT * (STRIDE) + array3_size(COUNT, STRIDE, sizeof(THING)) , ...) | kzalloc( - sizeof(THING) * COUNT * STRIDE + array3_size(COUNT, STRIDE, sizeof(THING)) , ...) ) // 3-factor product with 2 sizeof(variable), with redundant parens removed. @@ expression THING1, THING2; identifier COUNT; type TYPE1, TYPE2; @@ ( kzalloc( - sizeof(TYPE1) * sizeof(TYPE2) * COUNT + array3_size(COUNT, sizeof(TYPE1), sizeof(TYPE2)) , ...) | kzalloc( - sizeof(TYPE1) * sizeof(THING2) * (COUNT) + array3_size(COUNT, sizeof(TYPE1), sizeof(TYPE2)) , ...) | kzalloc( - sizeof(THING1) * sizeof(THING2) * COUNT + array3_size(COUNT, sizeof(THING1), sizeof(THING2)) , ...) | kzalloc( - sizeof(THING1) * sizeof(THING2) * (COUNT) + array3_size(COUNT, sizeof(THING1), sizeof(THING2)) , ...) | kzalloc( - sizeof(TYPE1) * sizeof(THING2) * COUNT + array3_size(COUNT, sizeof(TYPE1), sizeof(THING2)) , ...) | kzalloc( - sizeof(TYPE1) * sizeof(THING2) * (COUNT) + array3_size(COUNT, sizeof(TYPE1), sizeof(THING2)) , ...) ) // 3-factor product, only identifiers, with redundant parens removed. @@ identifier STRIDE, SIZE, COUNT; @@ ( kzalloc( - (COUNT) * STRIDE * SIZE + array3_size(COUNT, STRIDE, SIZE) , ...) | kzalloc( - COUNT * (STRIDE) * SIZE + array3_size(COUNT, STRIDE, SIZE) , ...) | kzalloc( - COUNT * STRIDE * (SIZE) + array3_size(COUNT, STRIDE, SIZE) , ...) | kzalloc( - (COUNT) * (STRIDE) * SIZE + array3_size(COUNT, STRIDE, SIZE) , ...) | kzalloc( - COUNT * (STRIDE) * (SIZE) + array3_size(COUNT, STRIDE, SIZE) , ...) | kzalloc( - (COUNT) * STRIDE * (SIZE) + array3_size(COUNT, STRIDE, SIZE) , ...) | kzalloc( - (COUNT) * (STRIDE) * (SIZE) + array3_size(COUNT, STRIDE, SIZE) , ...) | kzalloc( - COUNT * STRIDE * SIZE + array3_size(COUNT, STRIDE, SIZE) , ...) ) // Any remaining multi-factor products, first at least 3-factor products, // when they're not all constants... @@ expression E1, E2, E3; constant C1, C2, C3; @@ ( kzalloc(C1 * C2 * C3, ...) | kzalloc( - (E1) * E2 * E3 + array3_size(E1, E2, E3) , ...) | kzalloc( - (E1) * (E2) * E3 + array3_size(E1, E2, E3) , ...) | kzalloc( - (E1) * (E2) * (E3) + array3_size(E1, E2, E3) , ...) | kzalloc( - E1 * E2 * E3 + array3_size(E1, E2, E3) , ...) ) // And then all remaining 2 factors products when they're not all constants, // keeping sizeof() as the second factor argument. @@ expression THING, E1, E2; type TYPE; constant C1, C2, C3; @@ ( kzalloc(sizeof(THING) * C2, ...) | kzalloc(sizeof(TYPE) * C2, ...) | kzalloc(C1 * C2 * C3, ...) | kzalloc(C1 * C2, ...) | - kzalloc + kcalloc ( - sizeof(TYPE) * (E2) + E2, sizeof(TYPE) , ...) | - kzalloc + kcalloc ( - sizeof(TYPE) * E2 + E2, sizeof(TYPE) , ...) | - kzalloc + kcalloc ( - sizeof(THING) * (E2) + E2, sizeof(THING) , ...) | - kzalloc + kcalloc ( - sizeof(THING) * E2 + E2, sizeof(THING) , ...) | - kzalloc + kcalloc ( - (E1) * E2 + E1, E2 , ...) | - kzalloc + kcalloc ( - (E1) * (E2) + E1, E2 , ...) | - kzalloc + kcalloc ( - E1 * E2 + E1, E2 , ...) ) Signed-off-by: Kees Cook <keescook@chromium.org>
2018-01-03Merge branch 'for-mingo' of git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu into core/rcuIngo Molnar1-2/+0
Pull RCU updates from Paul E. McKenney: - Updates to use cond_resched() instead of cond_resched_rcu_qs() where feasible (currently everywhere except in kernel/rcu and in kernel/torture.c). Also a couple of fixes to avoid sending IPIs to offline CPUs. - Updates to simplify RCU's dyntick-idle handling. - Updates to remove almost all uses of smp_read_barrier_depends() and read_barrier_depends(). - Miscellaneous fixes. - Torture-test updates. Signed-off-by: Ingo Molnar <mingo@kernel.org>
2017-12-05drivers/dma/ioat: Remove now-redundant smp_read_barrier_depends()Paul E. McKenney1-2/+0
Now that READ_ONCE() implies smp_read_barrier_depends(), the __cleanup() and ioat_abort_descs() functions no longer need their smp_read_barrier_depends() calls, which this commit removes. It is actually not entirely clear why this driver ever included smp_read_barrier_depends() given that it appears to be x86-only and given that smp_read_barrier_depends() has no effect whatsoever except on DEC Alpha. Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Cc: Vinod Koul <vinod.koul@intel.com> Cc: Dan Williams <dan.j.williams@intel.com> Cc: <dmaengine@vger.kernel.org>
2017-11-29dmaengine: ioat: Fix error handling pathChristophe JAILLET1-1/+1
If the last test in 'ioat_dma_self_test()' fails, we must release all the allocated resources and not just part of them. Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr> Acked-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2017-10-24dmaengine: Convert timers to use timer_setup()Kees Cook3-6/+5
In preparation for unconditionally passing the struct timer_list pointer to all timer callbacks, switch to using the new timer_setup() and from_timer() to pass the timer pointer explicitly. Signed-off-by: Kees Cook <keescook@chromium.org> Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2017-08-23dmaengine: ioatdma: Add intr_coalesce sysfs entryUjjal Singh3-3/+52
We observed performance increase with DMA copy from memory to MMIO by changing the interrupt coalescing value to 0. The previous set value was projected on the C5xxx Xeon platform and no longer holds true. Removing hard coded value and providing a tune-able in sysfs in order to allow user to tune this on a per channel basis. By default this value will be set to 0. Example of sysfs variable importing for interrupt coalescing value from command line: echo 5> /sys/devices/pci0000:00/0000:00:04.0/dma/dma0chan0/ quickdata/intr_coalesce Reported-by: Nithin Sujir <nsujir@tintri.com> Signed-off-by: Ujjal Singh <ujjal.singh@intel.com> Acked-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2017-07-18dmaengine: ioat: constify pci_device_id.Arvind Yadav1-1/+1
pci_device_id are not supposed to change at runtime. All functions working with pci_device_id provided by <linux/pci.h> work with const pci_device_id. So mark the non-const structs as const. File size before: text data bss dec hex filename 12582 3056 16 15654 3d26 drivers/dma/ioat/init.o File size After adding 'const': text data bss dec hex filename 14773 865 16 15654 3d26 drivers/dma/ioat/init.o Signed-off-by: Arvind Yadav <arvind.yadav.cs@gmail.com> Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2017-07-08Merge tag 'dmaengine-4.13-rc1' of git://git.infradead.org/users/vkoul/slave-dmaLinus Torvalds1-4/+4
Pull dmaengine updates from Vinod Koul: - removal of AVR32 support in dw driver as AVR32 is gone - new driver for Broadcom stream buffer accelerator (SBA) RAID driver - add support for Faraday Technology FTDMAC020 in amba-pl08x driver - IOMMU support in pl330 driver - updates to bunch of drivers * tag 'dmaengine-4.13-rc1' of git://git.infradead.org/users/vkoul/slave-dma: (36 commits) dmaengine: qcom_hidma: correct API violation for submit dmaengine: zynqmp_dma: Remove max len check in zynqmp_dma_prep_memcpy dmaengine: tegra-apb: Really fix runtime-pm usage dmaengine: fsl_raid: make of_device_ids const. dmaengine: qcom_hidma: allow ACPI/DT parameters to be overridden dmaengine: fsldma: set BWC, DAHTS and SAHTS values correctly dmaengine: Kconfig: Simplify the help text for MXS_DMA dmaengine: pl330: Delete unused functions dmaengine: Replace WARN_TAINT_ONCE() with pr_warn_once() dmaengine: Kconfig: Extend the dependency for MXS_DMA dmaengine: mxs: Use %zu for printing a size_t variable dmaengine: ste_dma40: Cleanup scatterlist layering violations dmaengine: imx-dma: cleanup scatterlist layering violations dmaengine: use proper name for the R-Car SoC dmaengine: imx-sdma: Fix compilation warning. dmaengine: imx-sdma: Handle return value of clk_prepare_enable dmaengine: pl330: Add IOMMU support to slave tranfers dmaengine: DW DMAC: Handle return value of clk_prepare_enable dmaengine: pl08x: use GENMASK() to create bitmasks dmaengine: pl08x: Add support for Faraday Technology FTDMAC020 ...
2017-06-20dmaengine: ioat: don't use DMA_ERROR_CODEChristoph Hellwig1-17/+7
DMA_ERROR_CODE is not a public API and will go away. Instead properly unwind based on the loop counter. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Dave Jiang <dave.jiang@intel.com> Acked-By: Vinod Koul <vinod.koul@intel.com>
2017-06-15dmaengine: Replace WARN_TAINT_ONCE() with pr_warn_once()Prarit Bhargava1-4/+4
The WARN_TAINT_ONCE() prints out a loud stack trace on broken BIOSes. The systems that have this problem are several years out of support and no longer have BIOS updates available. The stack trace isn't necessary and a pr_warn_once() will do. Change WARN_TAINT_ONCE() to pr_warn_once() and taint. Signed-off-by: Prarit Bhargava <prarit@redhat.com> Cc: Dan Williams <dan.j.williams@intel.com> Cc: Vinod Koul <vinod.koul@intel.com> Cc: Duyck, Alexander H <alexander.h.duyck@intel.com> Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2017-04-24dmaengine: ioat: use setup_timerGeliang Tang1-3/+1
Use setup_timer() instead of init_timer() to simplify the code. Signed-off-by: Geliang Tang <geliangtang@gmail.com> Reviewed-by: Dan Williams <dan.j.williams@intel.com> Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2017-01-10dmaengine: iota: ioat_alloc_chan_resources should not perform sleeping allocations.Krister Johansen1-2/+2
On a kernel with DEBUG_LOCKS, ioat_free_chan_resources triggers an in_interrupt() warning. With PROVE_LOCKING, it reports detecting a SOFTIRQ-safe to SOFTIRQ-unsafe lock ordering in the same code path. This is because dma_generic_alloc_coherent() checks if the GFP flags permit blocking. It allocates from different subsystems if blocking is permitted. The free path knows how to return the memory to the correct allocator. If GFP_KERNEL is specified then the alloc and free end up going through cma_alloc(), which uses mutexes. Given that ioat_free_chan_resources() can be called in interrupt context, ioat_alloc_chan_resources() must specify GFP_NOWAIT so that the allocations do not block and instead use an allocator that uses spinlocks. Signed-off-by: Krister Johansen <kjlx@templeofstupid.com> Acked-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2017-01-02dmaengine: ioatdma: workaround SKX ioatdma versionDave Jiang1-0/+2
The Skylake ioatdma is technically CBDMA 3.2+ and contains the same hardware bits with some additional 3.3 features, but it's not really 3.3 where the driver is concerned. Signed-off-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2017-01-02dmaengine: ioatdma: Add Skylake PCI Dev IDDave Jiang2-1/+10
Adding Skylake Xeon PCI device ids for ioatdma and related bits. Signed-off-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2016-12-14Merge tag 'dmaengine-4.10-rc1' of git://git.infradead.org/users/vkoul/slave-dmaLinus Torvalds2-20/+18
Pull dmaengine updates from Vinod Koul: "Fairly routine update this time around with all changes specific to drivers: - New driver for STMicroelectronics FDMA - Memory-to-memory transfers on dw dmac - Support for slave maps on pl08x devices - Bunch of driver fixes to use dma_pool_zalloc - Bunch of compile and warning fixes spread across drivers" [ The ST FDMA driver already came in earlier through the remoteproc tree ] * tag 'dmaengine-4.10-rc1' of git://git.infradead.org/users/vkoul/slave-dma: (68 commits) dmaengine: sirf-dma: remove unused ‘sdesc’ dmaengine: pl330: remove unused ‘regs’ dmaengine: s3c24xx: remove unused ‘cdata’ dmaengine: stm32-dma: remove unused ‘src_addr’ dmaengine: stm32-dma: remove unused ‘dst_addr’ dmaengine: stm32-dma: remove unused ‘sfcr’ dmaengine: pch_dma: remove unused ‘cookie’ dmaengine: mic_x100_dma: remove unused ‘data’ dmaengine: img-mdc: remove unused ‘prev_phys’ dmaengine: usb-dmac: remove unused ‘uchan’ dmaengine: ioat: remove unused ‘res’ dmaengine: ioat: remove unused ‘ioat_dma’ dmaengine: ioat: remove unused ‘is_raid_device’ dmaengine: pl330: do not generate unaligned access dmaengine: k3dma: move to dma_pool_zalloc dmaengine: at_hdmac: move to dma_pool_zalloc dmaengine: at_xdmac: don't restore unsaved status dmaengine: ioat: set error code on failures dmaengine: ioat: set error code on failures dmaengine: DW DMAC: add multi-block property to device tree ...
2016-12-14Merge branch 'topic/ioat' into for-linusVinod Koul2-11/+18
2016-12-12dmaengine: ioat: remove unused ‘res’Vinod Koul1-3/+0
In __cleanup(), variable ‘res’ is initialized but never used, which leads to warning with W=1 drivers/dma/ioat/dma.c: In function ‘__cleanup’: drivers/dma/ioat/dma.c:614:28: warning: variable ‘res’ set but not used [-Wunused-but-set-variable] struct dmaengine_result res; So remove it. Cc: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2016-12-12dmaengine: ioat: remove unused ‘ioat_dma’Vinod Koul1-3/+0
In ioat_tx_submit_unlock(), variable ‘ioat_dma’ is initialized but never used, which leads to warning with W=1 drivers/dma/ioat/dma.c: In function ‘ioat_alloc_ring_ent’: drivers/dma/ioat/dma.c:341:25: warning: variable ‘ioat_dma’ set but not used [-Wunused-but-set-variable] struct ioatdma_device *ioat_dma; So remove it. Cc: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2016-12-12dmaengine: ioat: remove unused ‘is_raid_device’Vinod Koul1-3/+0
In ioat3_dma_probe(), variable ‘is_raid_device’ is initialized but never used, which leads to warning with W=1 drivers/dma/ioat/init.c: In function ‘ioat3_dma_probe’: drivers/dma/ioat/init.c:1084:7: warning: variable ‘is_raid_device’ set but not used [-Wunused-but-set-variable] bool is_raid_device = false; So remove it. Cc: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2016-12-06dmaengine: ioat: set error code on failuresPan Bian1-4/+12
In function ioat_xor_val_self_test(), when the calls to dma_mapping_error() fail, the value of return variable err is 0 (indicates no error). As a result, the return value may be inconsistent with the execution status. This patch fixes the bug by assigning "-ENOMEM" to err on the error path. Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=188601 Signed-off-by: Pan Bian <bianpan2016@163.com> Acked-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2016-12-06dmaengine: ioat: set error code on failuresPan Bian1-0/+2
In function ioat_dma_self_test(), when the calls to dma_mapping_error() fails, the value of return variable err is 0 (indicates no error). As a result, the return value may be inconsistent with the execution status. This patch fixes the bug by assigning -ENOMEM to err on the error path. Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=188591 Signed-off-by: Pan Bian <bianpan2016@163.com> Acked-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2016-11-18i7300_idle: Remove this driverLen Brown1-2/+0
In preparation for removing the idle_notifier, remove its only user, the i7300_idle driver. i7300_idle was deployed in 2008 to reduce idle memory power on systems using the i7300 chipset. The driver worked by throttling the fully-buffered DIMMs during idle periods using the IOAT DMA engine. The driver ran only on the i7300 chip-set, and no other hardware has used this mechanism. The driver no longer has a maintainer. Removing this driver will increase idle power on i7300 systems when they run the new kernel without the driver. Signed-off-by: Len Brown <len.brown@intel.com> Acked-by: Ingo Molnar <mingo@kernel.org> Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: http://lkml.kernel.org/r/ad6a044e57cc75f44cc8621abe846e58f7882243.1479449716.git.len.brown@intel.com Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2016-11-14dmaengine: ioatdma: error string table missing an entryDave Jiang1-0/+1
The error for DMA Transfer Source Address Error was missing. Signed-off-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2016-10-19dmaengine: ioatdma: loop for number elements in array chanerr_strColin Ian King1-7/+3
Just iterate over the number of elements in array chanerr_str rather than for all 32 bits. This removes the need for a NULL chanerr_str[i] check which could possibly overrun if the upper bits (28..31) of chanerr are set and 27th bit in chanerr is zero. This simplifies the code by removing an if statement and a break. Signed-off-by: Colin Ian King <colin.king@canonical.com> Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2016-10-03Merge branch 'topic/ioatdma' into for-linusVinod Koul1-1/+1
2016-08-08dmaengine: ioatdma: add error strings to chanerr outputDave Jiang1-4/+61
Provide a mechanism to translate CHANERR bits to English strings in order to allow user to report more concise errors. Signed-off-by: Dave Jiang <dave.jiang@intel.com> Reviewed-by: Lars-Peter Clausen <lars@metafoo.de> Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2016-08-08dmaengine: ioatdma: Add error handling to ioat driverDave Jiang2-16/+126
Adding error handling to the ioatdma driver so that when a read/write error occurs the error results are reported back and all the remaining descriptors are aborted. This utilizes the new dmaengine callback function that allows reporting of results. Signed-off-by: Dave Jiang <dave.jiang@intel.com> Reviewed-by: Lars-Peter Clausen <lars@metafoo.de> Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2016-08-08dmaengine: ioatdma: convert callback to helper functionDave Jiang1-8/+4
This is in preperation of moving to a callback that provides results to the callback for the transaction. The conversion will maintain current behavior and the driver must convert to new callback mechanism at a later time in order to receive results. Signed-off-by: Dave Jiang <dave.jiang@intel.com> Reviewed-by: Lars-Peter Clausen <lars@metafoo.de> Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2016-08-08dmaengine: ioatdma: fix uninitialized array usageDave Jiang1-1/+1
Static analysis showed that unitialized array is being used for compare. At line 850 when a dma_mapping_error() occurs, it jumps to dma_unmap. At this point, dma_srcs has not been initialized. However, the code after dma_unmap label checks dma_srcs for a comparison and thus is comparing to random garbage in the array. Given that when dest_dma is being mapped this is the first instance of mapping DMA memory and failed, there is really nothing to be cleaned up and thus should jump to free_resources label instead. Signed-off-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2016-07-23dmaengine: ioat: statify symbolVinod Koul1-1/+1
Sparse warns: drivers/dma/ioat/init.c:1215:6: warning: symbol 'ioat_resume' was not declared. Should it be static? Signed-off-by: Vinod Koul <vinod.koul@intel.com> Acked-by: Dave Jiang <dave.jiang@intel.com>
2016-05-14dmaengine: ioatdma: disable relaxed ordering for ioatdmaDave Jiang2-0/+19
ioatdma by default is in snoop mode. Relaxed ordering according to spec does not do anything in snoop mode. However, it causes hang or significant performance degrade when tested with NTB. Disabling in the driver due to some BIOS do not configure it correctly. Signed-off-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2016-05-03dmaengine: ioatdma: Use dma_pool_zallocJulia Lawall1-3/+2
Dma_pool_zalloc combines dma_pool_alloc and memset 0. The semantic patch that makes this transformation is as follows: (http://coccinelle.lip6.fr/) // <smpl> @@ expression d,e; statement S; @@ d = - dma_pool_alloc + dma_pool_zalloc (...); if (!d) S - memset(d, 0, sizeof(*d)); // </smpl> Signed-off-by: Julia Lawall <Julia.Lawall@lip6.fr> Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2016-03-17Merge tag 'dmaengine-4.6-rc1' of git://git.infradead.org/users/vkoul/slave-dmaLinus Torvalds5-235/+109
Pull dmaengine updates from Vinod Koul: "This is smallish update with minor changes to core and new driver and usual updates. Nothing super exciting here.. - We have made slave address as physical to enable driver to do the mapping. - We now expose the maxburst for slave dma as new capability so clients can know this and program accordingly - addition of device synchronize callbacks on omap and edma. - pl330 updates to support DMAFLUSHP for Rockchip platforms. - Updates and improved sg handling in Xilinx VDMA driver. - New hidma qualcomm dma driver, though some bits are still in progress" * tag 'dmaengine-4.6-rc1' of git://git.infradead.org/users/vkoul/slave-dma: (40 commits) dmaengine: IOATDMA: revise channel reset workaround on CB3.3 platforms dmaengine: add Qualcomm Technologies HIDMA channel driver dmaengine: add Qualcomm Technologies HIDMA management driver dmaengine: hidma: Add Device Tree binding dmaengine: qcom_bam_dma: move to qcom directory dmaengine: tegra: Move of_device_id table near to its user dmaengine: xilinx_vdma: Remove unnecessary variable initializations dmaengine: sirf: use __maybe_unused to hide pm functions dmaengine: rcar-dmac: clear pertinence number of channels dmaengine: sh: shdmac: don't open code of_device_get_match_data() dmaengine: tegra: don't open code of_device_get_match_data() dmaengine: qcom_bam_dma: Make driver work for BE dmaengine: sun4i: support module autoloading dma/mic_x100_dma: IS_ERR() vs PTR_ERR() typo dmaengine: xilinx_vdma: Use readl_poll_timeout instead of do while loop's dmaengine: xilinx_vdma: Simplify spin lock handling dmaengine: xilinx_vdma: Fix issues with non-parking mode dmaengine: xilinx_vdma: Improve SG engine handling dmaengine: pl330: fix to support the burst mode dmaengine: make slave address physical ...
2016-03-14Merge branch 'topic/ioatdma' into for-linusVinod Koul4-234/+108
2016-03-11dmaengine: IOATDMA: revise channel reset workaround on CB3.3 platformsDave Jiang2-36/+19
Previously we unloaded the interrupts and reloaded in order to work around a channel reset bug that cleared the MSIX table. This approach just isn't practical when a reset needs to happen in the error handler that just happens to be running in interrupt context (bottom half). It looks like we can work around the hardware issue by just storing a shadow copy of the MSIX table and restore it after reset. Signed-off-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2016-02-15dmaengine: IOATDMA: Allocate DMA descriptor ring in contig DMA memoryDave Jiang4-34/+81
Future IOATDMA hardware will take advantage of descriptors residing in contiguous memory. Setting the descriptor ring in max config DMA memory of 2MB. Each channel will need 2 of these chunks. This should provide 64k of 64B descriptors. Signed-off-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2016-02-15dmaengine: IOATDMA: Removing descriptor ring reshapeDave Jiang3-157/+1
Moving to contingous memory backed descriptor rings. This makes is really difficult and complex to do reshape. Going to remove this as I don't think we need to do it anymore. Signed-off-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2016-02-15dmaengine: IOATDMA: Convert pci_pool_* to dma_pool_*Dave Jiang3-15/+15
Converting old pci_pool_* calls to "new" dma_pool_* to make everything uniform. Signed-off-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2016-01-30dmaengine: IOATDMA: fix timer code that continues to restart channels during idleDave Jiang1-12/+22
The timer_event() function seems to have a bug where it ends up marking the last entry as non-responding and eventually attempts to restart the channel. This also continuously happen when idle. What needs to happen is for us to make sure there are no descriptors active and then handle that case properly. We should only hit the "cleanup" stage if there are still active descriptors. Signed-off-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2016-01-25dmaengine: ioatdma: Squelch framesize warningsTim Gardner1-1/+1
CC [M] drivers/dma/ioat/prep.o drivers/dma/ioat/prep.c: In function 'ioat_prep_pqxor': drivers/dma/ioat/prep.c:682:1: warning: the frame size of 1048 bytes is larger than 1024 bytes [-Wframe-larger-than=] } ^ drivers/dma/ioat/prep.c: In function 'ioat_prep_pqxor_val': drivers/dma/ioat/prep.c:714:1: warning: the frame size of 1048 bytes is larger than 1024 bytes [-Wframe-larger-than=] } gcc version 5.3.1 20151219 (Ubuntu 5.3.1-4ubuntu1) Cc: Vinod Koul <vinod.koul@intel.com> Cc: Dan Williams <dan.j.williams@intel.com> Cc: Dave Jiang <dave.jiang@intel.com> Cc: Prarit Bhargava <prarit@redhat.com> Cc: Nicholas Mc Guire <der.herr@hofr.at> Cc: Jarkko Nikula <jarkko.nikula@linux.intel.com> Signed-off-by: Tim Gardner <tim.gardner@canonical.com> Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2015-11-16dmaengine: ioatdma: constify dca_ops structuresJulia Lawall1-1/+1
The dca_ops structure is never modified, so declare it as const. Done with the help of Coccinelle. Signed-off-by: Julia Lawall <Julia.Lawall@lip6.fr> Acked-by: Dan Williams <dan.j.williams@intel.com> Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Copyright © 1996 – 2023 Jason A. Donenfeld. All Rights Reverse Engineered.