aboutsummaryrefslogtreecommitdiffstats
path: root/drivers/iommu (follow)
AgeCommit message (Collapse)AuthorFilesLines
2016-11-08iommu/vt-d: Fix dead-locks in disable_dmar_iommu() pathJoerg Roedel1-2/+12
It turns out that the disable_dmar_iommu() code-path tried to get the device_domain_lock recursivly, which will dead-lock when this code runs on dmar removal. Fix both code-paths that could lead to the dead-lock. Fixes: 55d940430ab9 ('iommu/vt-d: Get rid of domain->iommu_lock') Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-11-08iommu/arm-smmu: Fix out-of-bounds dereferenceRobin Murphy1-1/+3
When we iterate a master's config entries, what we generally care about is the entry's stream map index, rather than the entry index itself, so it's nice to have the iterator automatically assign the former from the latter. Unfortunately, booting with KASAN reveals the oversight that using a simple comma operator results in the entry index being dereferenced before being checked for validity, so we always access one element past the end of the fwspec array. Flip things around so that the check always happens before the index may be dereferenced. Fixes: adfec2e709d2 ("iommu/arm-smmu: Convert to iommu_fwspec") Reported-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-11-08iommu/arm-smmu: Check that iommu_fwspecs are oursRobin Murphy1-1/+1
We seem to have forgotten to check that iommu_fwspecs actually belong to us before we go ahead and dereference their private data. Oops. Fixes: 021bb8420d44 ("iommu/arm-smmu: Wire up generic configuration support") Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-11-08iommu/arm-smmu: Don't inadvertently reject multiple SMMUv3sRobin Murphy1-8/+17
We now delay installing our per-bus iommu_ops until we know an SMMU has successfully probed, as they don't serve much purpose beforehand, and doing so also avoids fights between multiple IOMMU drivers in a single kernel. However, the upshot of passing the return value of bus_set_iommu() back from our probe function is that if there happens to be more than one SMMUv3 device in a system, the second and subsequent probes will wind up returning -EBUSY to the driver core and getting torn down again. Avoid re-setting ops if ours are already installed, so that any genuine failures stand out. Fixes: 08d4ca2a672b ("iommu/arm-smmu: Support non-PCI devices with SMMUv3") CC: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> CC: Hanjun Guo <hanjun.guo@linaro.org> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-11-08iommu/arm-smmu: Work around ARM DMA configurationRobin Murphy1-0/+10
The 32-bit ARM DMA configuration code predates the IOMMU core's default domain functionality, and instead relies on allocating its own domains and attaching any devices using the generic IOMMU binding to them. Unfortunately, it does this relatively early on in the creation of the device, before we've seen our add_device callback, which leads us to attempt to operate on a half-configured master. To avoid a crash, check for this situation on attach, but refuse to play, as there's nothing we can do. This at least allows VFIO to keep working for people who update their 32-bit DTs to the generic binding, albeit with a few (innocuous) warnings from the DMA layer on boot. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-10-11Merge tag 'iommu-updates-v4.9' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommuLinus Torvalds13-848/+1118
Pull IOMMU updates from Joerg Roedel: - support for interrupt virtualization in the AMD IOMMU driver. These patches were shared with the KVM tree and are already merged through that tree. - generic DT-binding support for the ARM-SMMU driver. With this the driver now makes use of the generic DMA-API code. This also required some changes outside of the IOMMU code, but these are acked by the respective maintainers. - more cleanups and fixes all over the place. * tag 'iommu-updates-v4.9' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: (40 commits) iommu/amd: No need to wait iommu completion if no dte irq entry change iommu/amd: Free domain id when free a domain of struct dma_ops_domain iommu/amd: Use standard bitmap operation to set bitmap iommu/amd: Clean up the cmpxchg64 invocation iommu/io-pgtable-arm: Check for v7s-incapable systems iommu/dma: Avoid PCI host bridge windows iommu/dma: Add support for mapping MSIs iommu/arm-smmu: Set domain geometry iommu/arm-smmu: Wire up generic configuration support Docs: dt: document ARM SMMU generic binding usage iommu/arm-smmu: Convert to iommu_fwspec iommu/arm-smmu: Intelligent SMR allocation iommu/arm-smmu: Add a stream map entry iterator iommu/arm-smmu: Streamline SMMU data lookups iommu/arm-smmu: Refactor mmu-masters handling iommu/arm-smmu: Keep track of S2CR state iommu/arm-smmu: Consolidate stream map entry state iommu/arm-smmu: Handle stream IDs more dynamically iommu/arm-smmu: Set PRIVCFG in stage 1 STEs iommu/arm-smmu: Support non-PCI devices with SMMUv3 ...
2016-10-06Merge tag 'kvm-4.9-1' of git://git.kernel.org/pub/scm/virt/kvm/kvmLinus Torvalds4-63/+752
Pull KVM updates from Radim Krčmář: "All architectures: - move `make kvmconfig` stubs from x86 - use 64 bits for debugfs stats ARM: - Important fixes for not using an in-kernel irqchip - handle SError exceptions and present them to guests if appropriate - proxying of GICV access at EL2 if guest mappings are unsafe - GICv3 on AArch32 on ARMv8 - preparations for GICv3 save/restore, including ABI docs - cleanups and a bit of optimizations MIPS: - A couple of fixes in preparation for supporting MIPS EVA host kernels - MIPS SMP host & TLB invalidation fixes PPC: - Fix the bug which caused guests to falsely report lockups - other minor fixes - a small optimization s390: - Lazy enablement of runtime instrumentation - up to 255 CPUs for nested guests - rework of machine check deliver - cleanups and fixes x86: - IOMMU part of AMD's AVIC for vmexit-less interrupt delivery - Hyper-V TSC page - per-vcpu tsc_offset in debugfs - accelerated INS/OUTS in nVMX - cleanups and fixes" * tag 'kvm-4.9-1' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (140 commits) KVM: MIPS: Drop dubious EntryHi optimisation KVM: MIPS: Invalidate TLB by regenerating ASIDs KVM: MIPS: Split kernel/user ASID regeneration KVM: MIPS: Drop other CPU ASIDs on guest MMU changes KVM: arm/arm64: vgic: Don't flush/sync without a working vgic KVM: arm64: Require in-kernel irqchip for PMU support KVM: PPC: Book3s PR: Allow access to unprivileged MMCR2 register KVM: PPC: Book3S PR: Support 64kB page size on POWER8E and POWER8NVL KVM: PPC: Book3S: Remove duplicate setting of the B field in tlbie KVM: PPC: BookE: Fix a sanity check KVM: PPC: Book3S HV: Take out virtual core piggybacking code KVM: PPC: Book3S: Treat VTB as a per-subcore register, not per-thread ARM: gic-v3: Work around definition of gic_write_bpr1 KVM: nVMX: Fix the NMI IDT-vectoring handling KVM: VMX: Enable MSR-BASED TPR shadow even if APICv is inactive KVM: nVMX: Fix reload apic access page warning kvmconfig: add virtio-gpu to config fragment config: move x86 kvm_guest.config to a common location arm64: KVM: Remove duplicating init code for setting VMID ARM: KVM: Support vgic-v3 ...
2016-10-04Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linuxLinus Torvalds1-2/+1
Pull s390 updates from Martin Schwidefsky: "The new features and main improvements in this merge for v4.9 - Support for the UBSAN sanitizer - Set HAVE_EFFICIENT_UNALIGNED_ACCESS, it improves the code in some places - Improvements for the in-kernel fpu code, in particular the overhead for multiple consecutive in kernel fpu users is recuded - Add a SIMD implementation for the RAID6 gen and xor operations - Add RAID6 recovery based on the XC instruction - The PCI DMA flush logic has been improved to increase the speed of the map / unmap operations - The time synchronization code has seen some updates And bug fixes all over the place" * 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux: (48 commits) s390/con3270: fix insufficient space padding s390/con3270: fix use of uninitialised data MAINTAINERS: update DASD maintainer s390/cio: fix accidental interrupt enabling during resume s390/dasd: add missing \n to end of dev_err messages s390/config: Enable config options for Docker s390/dasd: make query host access interruptible s390/dasd: fix panic during offline processing s390/dasd: fix hanging offline processing s390/pci_dma: improve lazy flush for unmap s390/pci_dma: split dma_update_trans s390/pci_dma: improve map_sg s390/pci_dma: simplify dma address calculation s390/pci_dma: remove dma address range check iommu/s390: simplify registration of I/O address translation parameters s390: migrate exception table users off module.h and onto extable.h s390: export header for CLP ioctl s390/vmur: fix irq pointer dereference in int handler s390/dasd: add missing KOBJ_CHANGE event for unformatted devices s390: enable UBSAN ...
2016-09-22iommu/s390: simplify registration of I/O address translation parametersSebastian Ott1-2/+1
When a new function is attached to an iommu domain we need to register I/O address translation parameters. Since commit 69eea95c ("s390/pci_dma: fix DMA table corruption with > 4 TB main memory") start_dma and end_dma correctly describe the range of usable I/O addresses. Simplify the code by using these values directly. Signed-off-by: Sebastian Ott <sebott@linux.vnet.ibm.com> Reviewed-by: Gerald Schaefer <gerald.schaefer@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2016-09-20Merge branches 'x86/amd', 'x86/vt-d', 'arm/exynos', 'arm/mediatek', 'arm/renesas' and 'arm/smmu' into nextJoerg Roedel11-870/+1129
2016-09-20iommu/amd: No need to wait iommu completion if no dte irq entry changeBaoquan He1-3/+3
This is a clean up. In get_irq_table() only if DTE entry is changed iommu_completion_wait() need be called. Otherwise no need to do it. Signed-off-by: Baoquan He <bhe@redhat.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-09-19iommu/amd: Free domain id when free a domain of struct dma_ops_domainBaoquan He1-0/+3
The current code missed freeing domain id when free a domain of struct dma_ops_domain. Signed-off-by: Baoquan He <bhe@redhat.com> Fixes: ec487d1a110a ('x86, AMD IOMMU: add domain allocation and deallocation functions') Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-09-19iommu/amd: Use standard bitmap operation to set bitmapBaoquan He1-1/+2
It will be more readable and safer than the old setting. Signed-off-by: Baoquan He <bhe@redhat.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-09-19iommu/amd: Clean up the cmpxchg64 invocationBaoquan He1-1/+2
Change it as it's designed for and keep it consistent with other places. Signed-off-by: Baoquan He <bhe@redhat.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-09-16iommu/io-pgtable-arm: Check for v7s-incapable systemsRobin Murphy1-0/+4
On machines with no 32-bit addressable RAM whatsoever, we shouldn't even touch the v7s format as it's never going to work. Fixes: e5fc9753b1a8 ("iommu/io-pgtable: Add ARMv7 short descriptor support") Reported-by: Eric Auger <eric.auger@redhat.com> Tested-by: Eric Auger <eric.auger@redhat.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16iommu/dma: Avoid PCI host bridge windowsRobin Murphy1-1/+24
With our DMA ops enabled for PCI devices, we should avoid allocating IOVAs which a host bridge might misinterpret as peer-to-peer DMA and lead to faults, corruption or other badness. To be safe, punch out holes for all of the relevant host bridge's windows when initialising a DMA domain for a PCI device. CC: Marek Szyprowski <m.szyprowski@samsung.com> CC: Inki Dae <inki.dae@samsung.com> Reported-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16iommu/dma: Add support for mapping MSIsRobin Murphy1-15/+121
When an MSI doorbell is located downstream of an IOMMU, attaching devices to a DMA ops domain and switching on translation leads to a rude shock when their attempt to write to the physical address returned by the irqchip driver faults (or worse, writes into some already-mapped buffer) and no interrupt is forthcoming. Address this by adding a hook for relevant irqchip drivers to call from their compose_msi_msg() callback, to swizzle the physical address with an appropriatly-mapped IOVA for any device attached to one of our DMA ops domains. Acked-by: Thomas Gleixner <tglx@linutronix.de> Acked-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16iommu/arm-smmu: Set domain geometryRobin Murphy2-0/+4
For non-aperture-based IOMMUs, the domain geometry seems to have become the de-facto way of indicating the input address space size. That is quite a useful thing from the users' perspective, so let's do the same. Reviewed-by: Eric Auger <eric.auger@redhat.com> Tested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16iommu/arm-smmu: Wire up generic configuration supportRobin Murphy1-60/+108
With everything else now in place, fill in an of_xlate callback and the appropriate registration to plumb into the generic configuration machinery, and watch everything just work. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16iommu/arm-smmu: Convert to iommu_fwspecRobin Murphy1-62/+78
In the final step of preparation for full generic configuration support, swap our fixed-size master_cfg for the generic iommu_fwspec. For the legacy DT bindings, the driver simply gets to act as its own 'firmware'. Farewell, arbitrary MAX_MASTER_STREAMIDS! Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16iommu/arm-smmu: Intelligent SMR allocationRobin Murphy1-78/+119
Stream Match Registers are one of the more awkward parts of the SMMUv2 architecture; there are typically never enough to assign one to each stream ID in the system, and configuring them such that a single ID matches multiple entries is catastrophically bad - at best, every transaction raises a global fault; at worst, they go *somewhere*. To address the former issue, we can mask ID bits such that a single register may be used to match multiple IDs belonging to the same device or group, but doing so also heightens the risk of the latter problem (which can be nasty to debug). Tackle both problems at once by replacing the simple bitmap allocator with something much cleverer. Now that we have convenient in-memory representations of the stream mapping table, it becomes straightforward to properly validate new SMR entries against the current state, opening the door to arbitrary masking and SMR sharing. Another feature which falls out of this is that with IDs shared by separate devices being automatically accounted for, simply associating a group pointer with the S2CR offers appropriate group allocation almost for free, so hook that up in the process. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16iommu/arm-smmu: Add a stream map entry iteratorRobin Murphy1-14/+12
We iterate over the SMEs associated with a master config quite a lot in various places, and are about to do so even more. Let's wrap the idiom in a handy iterator macro before the repetition gets out of hand. Tested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16iommu/arm-smmu: Streamline SMMU data lookupsRobin Murphy1-33/+11
Simplify things somewhat by stashing our arm_smmu_device instance in drvdata, so that it's readily available to our driver model callbacks. Then we can excise the private list entirely, since the driver core already has a perfectly good list of SMMU devices we can use in the one instance we actually need to. Finally, make a further modest code saving with the relatively new of_device_get_match_data() helper. Tested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16iommu/arm-smmu: Refactor mmu-masters handlingRobin Murphy1-269/+101
To be able to support the generic bindings and handle of_xlate() calls, we need to be able to associate SMMUs and stream IDs directly with devices *before* allocating IOMMU groups. Furthermore, to support real default domains with multi-device groups we also have to handle domain attach on a per-device basis, as the "whole group at a time" assumption fails to properly handle subsequent devices added to a group after the first has already triggered default domain creation and attachment. To that end, use the now-vacant dev->archdata.iommu field for easy config and SMMU instance lookup, and unify config management by chopping down the platform-device-specific tree and probing the "mmu-masters" property on-demand instead. This may add a bit of one-off overhead to initially adding a new device, but we're about to deprecate that binding in favour of the inherently-more-efficient generic ones anyway. For the sake of simplicity, this patch does temporarily regress the case of aliasing PCI devices by losing the duplicate stream ID detection that the previous per-group config had. Stay tuned, because we'll be back to fix that in a better and more general way momentarily... Tested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16iommu/arm-smmu: Keep track of S2CR stateRobin Murphy1-66/+93
Making S2CRs first-class citizens within the driver with a high-level representation of their state offers a neat solution to a few problems: Firstly, the information about which context a device's stream IDs are associated with is already present by necessity in the S2CR. With that state easily accessible we can refer directly to it and obviate the need to track an IOMMU domain in each device's archdata (its earlier purpose of enforcing correct attachment of multi-device groups now being handled by the IOMMU core itself). Secondly, the core API now deprecates explicit domain detach and expects domain attach to move devices smoothly from one domain to another; for SMMUv2, this notion maps directly to simply rewriting the S2CRs assigned to the device. By giving the driver a suitable abstraction of those S2CRs to work with, we can massively reduce the overhead of the current heavy-handed "detach, free resources, reallocate resources, attach" approach. Thirdly, making the software state hardware-shaped and attached to the SMMU instance once again makes suspend/resume of this register group that much simpler to implement in future. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16iommu/arm-smmu: Consolidate stream map entry stateRobin Murphy1-61/+86
In order to consider SMR masking, we really want to be able to validate ID/mask pairs against existing SMR contents to prevent stream match conflicts, which at best would cause transactions to fault unexpectedly, and at worst lead to silent unpredictable behaviour. With our SMMU instance data holding only an allocator bitmap, and the SMR values themselves scattered across master configs hanging off devices which we may have no way of finding, there's essentially no way short of digging everything back out of the hardware. Similarly, the thought of power management ops to support suspend/resume faces the exact same problem. By massaging the software state into a closer shape to the underlying hardware, everything comes together quite nicely; the allocator and the high-level view of the data become a single centralised state which we can easily keep track of, and to which any updates can be validated in full before being synchronised to the hardware itself. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16iommu/arm-smmu: Handle stream IDs more dynamicallyRobin Murphy1-21/+22
Rather than assuming fixed worst-case values for stream IDs and SMR masks, keep track of whatever implemented bits the hardware actually reports. This also obviates the slightly questionable validation of SMR fields in isolation - rather than aborting the whole SMMU probe for a hardware configuration which is still architecturally valid, we can simply refuse masters later if they try to claim an unrepresentable ID or mask (which almost certainly implies a DT error anyway). Acked-by: Will Deacon <will.deacon@arm.com> Tested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16iommu/arm-smmu: Set PRIVCFG in stage 1 STEsRobin Murphy1-1/+6
Implement the SMMUv3 equivalent of d346180e70b9 ("iommu/arm-smmu: Treat all device transactions as unprivileged"), so that once again those pesky DMA controllers with their privileged instruction fetches don't unexpectedly fault in stage 1 domains due to VMSAv8 rules. Acked-by: Will Deacon <will.deacon@arm.com> Tested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16iommu/arm-smmu: Support non-PCI devices with SMMUv3Robin Murphy2-7/+32
With the device <-> stream ID relationship suitably abstracted and of_xlate() hooked up, the PCI dependency now looks, and is, entirely arbitrary. Any bus using the of_dma_configure() mechanism will work, so extend support to the platform and AMBA buses which do just that. Acked-by: Will Deacon <will.deacon@arm.com> Tested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16iommu/arm-smmu: Implement of_xlate() for SMMUv3Robin Murphy1-170/+128
Now that we can properly describe the mapping between PCI RIDs and stream IDs via "iommu-map", and have it fed it to the driver automatically via of_xlate(), rework the SMMUv3 driver to benefit from that, and get rid of the current misuse of the "iommus" binding. Since having of_xlate wired up means that masters will now be given the appropriate DMA ops, we also need to make sure that default domains work properly. This necessitates dispensing with the "whole group at a time" notion for attaching to a domain, as devices which share a group get attached to the group's default domain one by one as they are initially probed. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16iommu/arm-smmu: Fall back to global bypassRobin Murphy1-4/+44
Unlike SMMUv2, SMMUv3 has no easy way to bypass unknown stream IDs, other than allocating and filling in the entire stream table with bypass entries, which for some configurations would waste *gigabytes* of RAM. Otherwise, all transactions on unknown stream IDs will simply be aborted with a C_BAD_STREAMID event. Rather than render the system unusable in the case of an invalid DT, avoid enabling the SMMU altogether such that everything bypasses (though letting the explicit disable_bypass option take precedence). Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16iommu: Introduce iommu_fwspecRobin Murphy2-2/+64
Introduce a common structure to hold the per-device firmware data that most IOMMU drivers need to keep track of. This enables us to configure much of that data from common firmware code, and consolidate a lot of the equivalent implementations, device look-up tables, etc. which are currently strewn across IOMMU drivers. This will also be enable us to address the outstanding "multiple IOMMUs on the platform bus" problem by tweaking IOMMU API calls to prefer dev->fwspec->ops before falling back to dev->bus->iommu_ops, and thus gracefully handle those troublesome systems which we currently cannot. As the first user, hook up the OF IOMMU configuration mechanism. The driver-defined nature of DT cells means that we still need the drivers to translate and add the IDs themselves, but future users such as the much less free-form ACPI IORT will be much simpler and self-contained. CC: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Suggested-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16iommu/of: Handle iommu-map property for PCIRobin Murphy1-5/+41
Now that we have a way to pick up the RID translation and target IOMMU, hook up of_iommu_configure() to bring PCI devices into the of_xlate mechanism and allow them IOMMU-backed DMA ops without the need for driver-specific handling. Reviewed-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16iommu/arm-smmu: Disable interrupts whilst holding the cmdq lockWill Deacon1-2/+3
The cmdq lock is taken whenever we issue commands into the command queue, which can occur in IRQ context (as a result of unmap) or in process context (as a result of a threaded IRQ handler or device probe). This can lead to a theoretical deadlock if the interrupt handler performing the unmap hits whilst the lock is taken, so explicitly use the {irqsave,irqrestore} spin_lock accessors for the cmdq lock. Tested-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16iommu/arm-smmu: Fix polling of command queueJean-Philippe Brucker1-19/+9
When the SMMUv3 driver attempts to send a command, it adds an entry to the command queue. This is a circular buffer, where both the producer and consumer have a wrap bit. When producer.index == consumer.index and producer.wrap == consumer.wrap, the list is empty. When producer.index == consumer.index and producer.wrap != consumer.wrap, the list is full. If the list is full when the driver needs to add a command, it waits for the SMMU to consume one command, and advance the consumer pointer. The problem is that we currently rely on "X before Y" operation to know if entries have been consumed, which is a bit fiddly since it only makes sense when the distance between X and Y is less than or equal to the size of the queue. At the moment when the list is full, we use "Consumer before Producer + 1", which is out of range and returns a value opposite to what we expect: when the queue transitions to not full, we stay in the polling loop and time out, printing an error. Given that the actual bug was difficult to determine, simplify the polling logic by relying exclusively on queue_full and queue_empty, that don't have this range constraint. Polling the queue is now straightforward: * When we want to add a command and the list is full, wait until it isn't full and retry. * After adding a sync, wait for the list to be empty before returning. Suggested-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16iommu/arm-smmu: Support v7s context formatRobin Murphy1-22/+47
Fill in the last bits of machinery required to drive a stage 1 context bank in v7 short descriptor format. By default we'll prefer to use it only when the CPUs are also using the same format, such that we're guaranteed that everything will be strictly 32-bit. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16iommu/arm-smmu: Fix event queues synchronizationJean-Philippe Brucker1-84/+66
SMMUv3 only sends interrupts for event queues (EVTQ and PRIQ) when they transition from empty to non-empty. At the moment, if the SMMU adds new items to a queue before the event thread finished consuming a previous batch, the driver ignores any new item. The queue is then stuck in non-empty state and all subsequent events will be lost. As an example, consider the following flow, where (P, C) is the SMMU view of producer/consumer indices, and (p, c) the driver view. P C | p c 1. SMMU appends a PPR to the PRI queue, 1 0 | 0 0 sends an MSI 2. PRIQ handler is called. 1 0 | 1 0 3. SMMU appends a PPR to the PRI queue. 2 0 | 1 0 4. PRIQ thread removes the first element. 2 1 | 1 1 5. PRIQ thread believes that the queue is empty, goes into idle indefinitely. To avoid this, always synchronize the producer index and drain the queue once before leaving an event handler. In order to prevent races on the local producer index, move all event queue handling into the threads. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16iommu/arm-smmu: Drop devm_free_irq when driver detachPeng Fan1-4/+0
There is no need to call devm_free_irq when driver detach. devres_release_all which is called after 'drv->remove' will release all managed resources. Signed-off-by: Peng Fan <van.freenix@gmail.com> Reviewed-by: Robin Murphy <robin.murphy@arm.com> Cc: Will Deacon <will.deacon@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-15iommu/amd: Don't put completion-wait semaphore on stackJoerg Roedel2-16/+37
The semaphore used by the AMD IOMMU to signal command completion lived on the stack until now, which was safe as the driver busy-waited on the semaphore with IRQs disabled, so the stack can't go away under the driver. But the recently introduced vmap-based stacks break this as the physical address of the semaphore can't be determinded easily anymore. The driver used the __pa() macro, but that only works in the direct-mapping. The result were Completion-Wait timeout errors seen by the IOMMU driver, breaking system boot. Since putting the semaphore on the stack is bad design anyway, move the semaphore into 'struct amd_iommu'. It is protected by the per-iommu lock and now in the direct mapping again. This fixes the Completion-Wait timeout errors and makes AMD IOMMU systems boot again with vmap-based stacks enabled. Reported-by: Borislav Petkov <bp@alien8.de> Signed-off-by: Joerg Roedel <jroedel@suse.de> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
2016-09-05Merge branch 'x86/amd-avic' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu into HEADPaolo Bonzini4-63/+752
Merge IOMMU bits for virtualization of interrupt injection into virtual machines.
2016-09-05iommu/vt-d: Make sure RMRRs are mapped before domain goes publicJoerg Roedel1-7/+20
When a domain is allocated through the get_valid_domain_for_dev path, it will be context-mapped before the RMRR regions are mapped in the page-table. This opens a short time window where device-accesses to these regions fail and causing DMAR faults. Fix this by mapping the RMRR regions before the domain is context-mapped. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-09-05iommu/vt-d: Split up get_domain_for_dev functionJoerg Roedel1-21/+55
Split out the search for an already existing domain and the context mapping of the device to the new domain. This allows to map possible RMRR regions into the domain before it is context mapped. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-09-05iommu/ipmmu-vmsa: Fix wrong error handle of ipmmu_add_deviceShawn Lin1-1/+1
Let's fix the error handle of ipmmu_add_device when failing to find utlbs, otherwise we take a risk of pontential memleak. Signed-off-by: Shawn Lin <shawn.lin@rock-chips.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-09-05Merge branch 'x86/amd-avic' into x86/amdJoerg Roedel4-63/+752
2016-09-05iommu/amd: Remove AMD_IOMMU_STATSFabian Frederick1-6/+0
Commit e85e8f69cedb ("iommu/amd: Remove statistics code") removed that configuration. Also remove function definition (suggested by Joerg Roedel) Signed-off-by: Fabian Frederick <fabf@skynet.be> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-09-05iommu/amd: Enable vAPIC interrupt remapping mode by defaultSuravee Suthikulpanit3-10/+48
Introduce struct iommu_dev_data.use_vapic flag, which IOMMU driver uses to determine if it should enable vAPIC support, by setting the ga_mode bit in the device's interrupt remapping table entry. Currently, it is enabled for all pass-through device if vAPIC mode is enabled. Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-09-05iommu/amd: Implements irq_set_vcpu_affinity() hook to setup vapic mode for pass-through devicesSuravee Suthikulpanit2-4/+65
This patch implements irq_set_vcpu_affinity() function to set up interrupt remapping table entry with vapic mode for pass-through devices. In case requirements for vapic mode are not met, it falls back to set up the IRTE in legacy mode. Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-09-05iommu/amd: Introduce amd_iommu_update_ga()Suravee Suthikulpanit2-0/+40
Introduces a new IOMMU API, amd_iommu_update_ga(), which allows KVM (SVM) to update existing posted interrupt IOMMU IRTE when load/unload vcpu. Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-09-05iommu/amd: Adding GALOG interrupt handlerSuravee Suthikulpanit1-3/+70
This patch adds AMD IOMMU guest virtual APIC log (GALOG) handler. When IOMMU hardware receives an interrupt targeting a blocking vcpu, it creates an entry in the GALOG, and generates an interrupt to notify the AMD IOMMU driver. At this point, the driver processes the log entry, and notify the SVM driver via the registered iommu_ga_log_notifier function. Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-09-05iommu/amd: Detect and initialize guest vAPIC logSuravee Suthikulpanit2-7/+133
This patch adds support to detect and initialize IOMMU Guest vAPIC log (GALOG). By default, it also enable GALog interrupt to notify IOMMU driver when GA Log entry is created. Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
Copyright © 1996 – 2023 Jason A. Donenfeld. All Rights Reverse Engineered.