aboutsummaryrefslogtreecommitdiffstats
path: root/drivers/iommu (follow)
AgeCommit message (Collapse)AuthorFilesLines
2020-02-05Merge tag 'iommu-updates-v5.6' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommuLinus Torvalds27-800/+1555
Pull iommu updates from Joerg Roedel: - Allow compiling the ARM-SMMU drivers as modules. - Fixes and cleanups for the ARM-SMMU drivers and io-pgtable code collected by Will Deacon. The merge-commit (6855d1ba7537) has all the details. - Cleanup of the iommu_put_resv_regions() call-backs in various drivers. - AMD IOMMU driver cleanups. - Update for the x2APIC support in the AMD IOMMU driver. - Preparation patches for Intel VT-d nested mode support. - RMRR and identity domain handling fixes for the Intel VT-d driver. - More small fixes and cleanups. * tag 'iommu-updates-v5.6' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: (87 commits) iommu/amd: Remove the unnecessary assignment iommu/vt-d: Remove unnecessary WARN_ON_ONCE() iommu/vt-d: Unnecessary to handle default identity domain iommu/vt-d: Allow devices with RMRRs to use identity domain iommu/vt-d: Add RMRR base and end addresses sanity check iommu/vt-d: Mark firmware tainted if RMRR fails sanity check iommu/amd: Remove unused struct member iommu/amd: Replace two consecutive readl calls with one readq iommu/vt-d: Don't reject Host Bridge due to scope mismatch PCI/ATS: Add PASID stubs iommu/arm-smmu-v3: Return -EBUSY when trying to re-add a device iommu/arm-smmu-v3: Improve add_device() error handling iommu/arm-smmu-v3: Use WRITE_ONCE() when changing validity of an STE iommu/arm-smmu-v3: Add second level of context descriptor table iommu/arm-smmu-v3: Prepare for handling arm_smmu_write_ctx_desc() failure iommu/arm-smmu-v3: Propagate ssid_bits iommu/arm-smmu-v3: Add support for Substream IDs iommu/arm-smmu-v3: Add context descriptor tables allocators iommu/arm-smmu-v3: Prepare arm_smmu_s1_cfg for SSID support ACPI/IORT: Parse SSID property of named component node ...
2020-01-31Merge tag 'pci-v5.6-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pciLinus Torvalds2-12/+6
Pull PCI updates from Bjorn Helgaas: "Resource management: - Improve resource assignment for hot-added nested bridges, e.g., Thunderbolt (Nicholas Johnson) Power management: - Optionally print config space of devices before suspend (Chen Yu) - Increase D3 delay for AMD Ryzen5/7 XHCI controllers (Daniel Drake) Virtualization: - Generalize DMA alias quirks (James Sewart) - Add DMA alias quirk for PLX PEX NTB (James Sewart) - Fix IOV memory leak (Navid Emamdoost) AER: - Log which device prevents error recovery (Yicong Yang) Peer-to-peer DMA: - Whitelist Intel SkyLake-E (Armen Baloyan) Broadcom iProc host bridge driver: - Apply PAXC quirk whether driver is built-in or module (Wei Liu) Broadcom STB host bridge driver: - Add Broadcom STB PCIe host controller driver (Jim Quinlan) Intel Gateway SoC host bridge driver: - Add driver for Intel Gateway SoC (Dilip Kota) Intel VMD host bridge driver: - Add support for DMA aliases on other buses (Jon Derrick) - Remove dma_map_ops overrides (Jon Derrick) - Remove now-unused X86_DEV_DMA_OPS (Christoph Hellwig) NVIDIA Tegra host bridge driver: - Fix Tegra30 afi_pex2_ctrl register offset (Marcel Ziswiler) Panasonic UniPhier host bridge driver: - Remove module code since driver can't be built as a module (Masahiro Yamada) Qualcomm host bridge driver: - Add support for SDM845 PCIe controller (Bjorn Andersson) TI Keystone host bridge driver: - Fix "num-viewport" DT property error handling (Kishon Vijay Abraham I) - Fix link training retries initiation (Yurii Monakov) - Fix outbound region mapping (Yurii Monakov) Misc: - Add Switchtec Gen4 support (Kelvin Cao) - Add Switchtec Intercomm Notify and Upstream Error Containment support (Logan Gunthorpe) - Use dma_set_mask_and_coherent() since Switchtec supports 64-bit addressing (Wesley Sheng)" * tag 'pci-v5.6-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci: (60 commits) PCI: Allow adjust_bridge_window() to shrink resource if necessary PCI: Set resource size directly in adjust_bridge_window() PCI: Rename extend_bridge_window() to adjust_bridge_window() PCI: Rename extend_bridge_window() parameter PCI: Consider alignment of hot-added bridges when assigning resources PCI: Remove local variable usage in pci_bus_distribute_available_resources() PCI: Pass size + alignment to pci_bus_distribute_available_resources() PCI: Rename variables PCI: vmd: Add two VMD Device IDs PCI: Remove unnecessary braces PCI: brcmstb: Add MSI support PCI: brcmstb: Add Broadcom STB PCIe host controller driver x86/PCI: Remove X86_DEV_DMA_OPS PCI: vmd: Remove dma_map_ops overrides iommu/vt-d: Remove VMD child device sanity check iommu/vt-d: Use pci_real_dma_dev() for mapping PCI: Introduce pci_real_dma_dev() x86/PCI: Expose VMD's pci_dev in struct pci_sysdata x86/PCI: Add to_pci_sysdata() helper PCI/AER: Initialize aer_fifo ...
2020-01-29Merge branch 'pci/host-vmd'Bjorn Helgaas1-7/+4
- Save VMD's pci_dev in x86 struct pci_sysdata (Jon Derrick) - Add pci_real_dma_dev() for DMA aliases not on the same bus as requester (Jon Derrick) - Add IOMMU mappings for pci_real_dma_dev() (Jon Derrick) - Remove IOMMU sanity checks for VMD devices (Jon Derrick) - Remove VMD dma_map_ops overrides (Jon Derrick) - Remove unused X86_DEV_DMA_OPS (Christoph Hellwig) - Add VMD device IDs that need bus restriction mode (Sushma Kalakota) * pci/host-vmd: PCI: vmd: Add two VMD Device IDs x86/PCI: Remove X86_DEV_DMA_OPS PCI: vmd: Remove dma_map_ops overrides iommu/vt-d: Remove VMD child device sanity check iommu/vt-d: Use pci_real_dma_dev() for mapping PCI: Introduce pci_real_dma_dev() x86/PCI: Expose VMD's pci_dev in struct pci_sysdata x86/PCI: Add to_pci_sysdata() helper
2020-01-27Merge tag 'ioremap-5.6' of git://git.infradead.org/users/hch/ioremapLinus Torvalds1-1/+1
Pull ioremap updates from Christoph Hellwig: "Remove the ioremap_nocache API (plus wrappers) that are always identical to ioremap" * tag 'ioremap-5.6' of git://git.infradead.org/users/hch/ioremap: remove ioremap_nocache and devm_ioremap_nocache MIPS: define ioremap_nocache to ioremap
2020-01-24iommu/vt-d: Remove VMD child device sanity checkJon Derrick1-9/+1
Remove the sanity check required for VMD child devices. The new pci_real_dma_dev() DMA alias mechanism places them in the same IOMMU group as the VMD endpoint. Assignment of the group would require assigning the VMD endpoint, where unbinding the VMD endpoint removes the child device domain from the hierarchy. Link: https://lore.kernel.org/r/1579613871-301529-6-git-send-email-jonathan.derrick@intel.com Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Lu Baolu <baolu.lu@linux.intel.com>
2020-01-24iommu/vt-d: Use pci_real_dma_dev() for mappingJon Derrick1-0/+5
The PCI device may have a DMA requester on another bus, such as VMD subdevices needing to use the VMD endpoint. This case requires the real DMA device for the IOMMU mapping, so use pci_real_dma_dev() to find that device. Link: https://lore.kernel.org/r/1579613871-301529-5-git-send-email-jonathan.derrick@intel.com Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Lu Baolu <baolu.lu@linux.intel.com>
2020-01-24Merge branches 'iommu/fixes', 'arm/smmu', 'x86/amd', 'x86/vt-d' and 'core' into nextJoerg Roedel27-800/+1555
2020-01-24iommu/amd: Remove the unnecessary assignmentAdrian Huang1-1/+0
The assignment of the global variable 'iommu_detected' has been moved from amd_iommu_init_dma_ops() to amd_iommu_detect(), so this patch removes the assignment in amd_iommu_init_dma_ops(). Signed-off-by: Adrian Huang <ahuang12@lenovo.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-01-24iommu/vt-d: Remove unnecessary WARN_ON_ONCE()Lu Baolu1-1/+0
Address field in device TLB invalidation descriptor is qualified by the S field. If S field is zero, a single page at page address specified by address [63:12] is requested to be invalidated. If S field is set, the least significant bit in the address field with value 0b (say bit N) indicates the invalidation address range. The spec doesn't require the address [N - 1, 0] to be cleared, hence remove the unnecessary WARN_ON_ONCE(). Otherwise, the caller might set "mask = MAX_AGAW_PFN_WIDTH" in order to invalidating all the cached mappings on an endpoint, and below overflow error will be triggered. [...] UBSAN: Undefined behaviour in drivers/iommu/dmar.c:1354:3 shift exponent 64 is too large for 64-bit type 'long long unsigned int' [...] Reported-and-tested-by: Frank <fgndev@posteo.de> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-01-24iommu/vt-d: Unnecessary to handle default identity domainLu Baolu1-7/+2
The iommu default domain framework has been designed to take care of setting identity default domain type. It's unnecessary to handle this again in the VT-d driver. Hence, remove it. Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-01-24iommu/vt-d: Allow devices with RMRRs to use identity domainLu Baolu1-13/+2
Since commit ea2447f700cab ("intel-iommu: Prevent devices with RMRRs from being placed into SI Domain"), the Intel IOMMU driver doesn't allow any devices with RMRR locked to use the identity domain. This was added to to fix the issue where the RMRR info for devices being placed in and out of the identity domain gets lost. This identity maps all RMRRs when setting up the identity domain, so that devices with RMRRs could also use it. Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-01-24iommu/vt-d: Add RMRR base and end addresses sanity checkBarret Rhoden1-1/+12
The VT-d spec specifies requirements for the RMRR entries base and end (called 'Limit' in the docs) addresses. This commit will cause the DMAR processing to mark the firmware as tainted if any RMRR entries that do not meet these requirements. Signed-off-by: Barret Rhoden <brho@google.com> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-01-24iommu/vt-d: Mark firmware tainted if RMRR fails sanity checkBarret Rhoden1-4/+8
RMRR entries describe memory regions that are DMA targets for devices outside the kernel's control. RMRR entries that fail the sanity check are pointing to regions of memory that the firmware did not tell the kernel are reserved or otherwise should not be used. Instead of aborting DMAR processing, this commit marks the firmware as tainted. These RMRRs will still be identity mapped, otherwise, some devices, e.x. graphic devices, will not work during boot. Signed-off-by: Barret Rhoden <brho@google.com> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Fixes: f036c7fa0ab60 ("iommu/vt-d: Check VT-d RMRR region in BIOS is reported as reserved") Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-01-24iommu/amd: Fix IOMMU perf counter clobbering during initShuah Khan1-6/+18
init_iommu_perf_ctr() clobbers the register when it checks write access to IOMMU perf counters and fails to restore when they are writable. Add save and restore to fix it. Signed-off-by: Shuah Khan <skhan@linuxfoundation.org> Fixes: 30861ddc9cca4 ("perf/x86/amd: Add IOMMU Performance Counter resource management") Reviewed-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Tested-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-01-24iommu/vt-d: Call __dmar_remove_one_dev_info with valid pointerJerry Snitselaar1-1/+2
It is possible for archdata.iommu to be set to DEFER_DEVICE_DOMAIN_INFO or DUMMY_DEVICE_DOMAIN_INFO so check for those values before calling __dmar_remove_one_dev_info. Without a check it can result in a null pointer dereference. This has been seen while booting a kdump kernel on an HP dl380 gen9. Cc: Joerg Roedel <joro@8bytes.org> Cc: Lu Baolu <baolu.lu@linux.intel.com> Cc: David Woodhouse <dwmw2@infradead.org> Cc: stable@vger.kernel.org # 5.3+ Cc: linux-kernel@vger.kernel.org Fixes: ae23bfb68f28 ("iommu/vt-d: Detach domain before using a private one") Signed-off-by: Jerry Snitselaar <jsnitsel@redhat.com> Acked-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-01-17iommu/amd: Remove unused struct memberAdrian Huang1-1/+0
Commit c805b428f206 ("iommu/amd: Remove amd_iommu_pd_list") removes the global list for the allocated protection domains. The corresponding member 'list' of the protection_domain struct is not used anymore, so it can be removed. Signed-off-by: Adrian Huang <ahuang12@lenovo.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-01-17iommu/amd: Replace two consecutive readl calls with one readqAdrian Huang1-5/+1
Optimize the reigster reading by using readq instead of the two consecutive readl calls. Signed-off-by: Adrian Huang <ahuang12@lenovo.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-01-17iommu/vt-d: Don't reject Host Bridge due to scope mismatchjimyan1-1/+1
On a system with two host bridges(0000:00:00.0,0000:80:00.0), iommu initialization fails with DMAR: Device scope type does not match for 0000:80:00.0 This is because the DMAR table reports this device as having scope 2 (ACPI_DMAR_SCOPE_TYPE_BRIDGE): but the device has a type 0 PCI header: 80:00.0 Class 0600: Device 8086:2020 (rev 06) 00: 86 80 20 20 47 05 10 00 06 00 00 06 10 00 00 00 10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 00 00 30: 00 00 00 00 90 00 00 00 00 00 00 00 00 01 00 00 VT-d works perfectly on this system, so there's no reason to bail out on initialization due to this apparent scope mismatch. Add the class 0x06 ("PCI_BASE_CLASS_BRIDGE") as a heuristic for allowing DMAR initialization for non-bridge PCI devices listed with scope bridge. Signed-off-by: jimyan <jimyan@baidu.com> Reviewed-by: Jerry Snitselaar <jsnitsel@redhat.com> Reviewed-by: Roland Dreier <roland@purestorage.com> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-01-15iommu/arm-smmu-v3: Return -EBUSY when trying to re-add a deviceWill Deacon1-21/+16
Although we WARN in arm_smmu_add_device() if the device being added has been added already without a subsequent call to arm_smmu_remove_device(), we still continue half-heartedly, initialising the stream-table for any new StreamIDs that may have magically appeared and re-establishing device links that should still be there from last time. Given that calling ->add_device() twice without removing the device in the meantime is indicative of an error in the caller, just return -EBUSY after warning. Cc: Robin Murphy <robin.murphy@arm.com> Cc: Jean Philippe-Brucker <jean-philippe@linaro.org> Signed-off-by: Will Deacon <will@kernel.org>
2020-01-15iommu/arm-smmu-v3: Improve add_device() error handlingJean-Philippe Brucker1-7/+21
Let add_device() clean up after itself. The iommu_bus_init() function does call remove_device() on error, but other sites (e.g. of_iommu) do not. Don't free level-2 stream tables because we'd have to track if we allocated each of them or if they are used by other endpoints. It's not worth the hassle since they are managed resources. Reviewed-by: Eric Auger <eric.auger@redhat.com> Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Signed-off-by: Will Deacon <will@kernel.org>
2020-01-15iommu/arm-smmu-v3: Use WRITE_ONCE() when changing validity of an STEWill Deacon1-1/+2
If, for some bizarre reason, the compiler decided to split up the write of STE DWORD 0, we could end up making a partial structure valid. Although this probably won't happen, follow the example of the context-descriptor code and use WRITE_ONCE() to ensure atomicity of the write. Reported-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Signed-off-by: Will Deacon <will@kernel.org>
2020-01-15iommu/arm-smmu-v3: Add second level of context descriptor tableJean-Philippe Brucker1-8/+134
The SMMU can support up to 20 bits of SSID. Add a second level of page tables to accommodate this. Devices that support more than 1024 SSIDs now have a table of 1024 L1 entries (8kB), pointing to tables of 1024 context descriptors (64kB), allocated on demand. Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Signed-off-by: Will Deacon <will@kernel.org>
2020-01-15iommu/arm-smmu-v3: Prepare for handling arm_smmu_write_ctx_desc() failureJean-Philippe Brucker1-4/+12
Second-level context descriptor tables will be allocated lazily in arm_smmu_write_ctx_desc(). Help with handling allocation failure by moving the CD write into arm_smmu_domain_finalise_s1(). Reviewed-by: Eric Auger <eric.auger@redhat.com> Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> [will: Add comment per discussion on list] Signed-off-by: Will Deacon <will@kernel.org>
2020-01-15iommu/arm-smmu-v3: Propagate ssid_bitsJean-Philippe Brucker1-3/+16
Now that we support substream IDs, initialize s1cdmax with the number of SSID bits supported by a master and the SMMU. Context descriptor tables are allocated once for the first master attached to a domain. Therefore attaching multiple devices with different SSID sizes is tricky, and we currently don't support it. As a future improvement it would be nice to at least support attaching a SSID-capable device to a domain that isn't using SSID, by reallocating the SSID table. This would allow supporting a SSID-capable device that is in the same IOMMU group as a bridge, for example. Varying SSID size is less of a concern, since the PCIe specification "highly recommends" that devices supporting PASID implement all 20 bits of it. Tested-by: Zhangfei Gao <zhangfei.gao@linaro.org> Reviewed-by: Eric Auger <eric.auger@redhat.com> Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Signed-off-by: Will Deacon <will@kernel.org>
2020-01-15iommu/arm-smmu-v3: Add support for Substream IDsJean-Philippe Brucker1-22/+111
At the moment, the SMMUv3 driver implements only one stage-1 or stage-2 page directory per device. However SMMUv3 allows more than one address space for some devices, by providing multiple stage-1 page directories. In addition to the Stream ID (SID), that identifies a device, we can now have Substream IDs (SSID) identifying an address space. In PCIe, SID is called Requester ID (RID) and SSID is called Process Address-Space ID (PASID). A complete stage-1 walk goes through the context descriptor table: Stream tables Ctx. Desc. tables Page tables +--------+ ,------->+-------+ ,------->+-------+ : : | : : | : : +--------+ | +-------+ | +-------+ SID->| STE |---' SSID->| CD |---' IOVA->| PTE |--> IPA +--------+ +-------+ +-------+ : : : : : : +--------+ +-------+ +-------+ Rewrite arm_smmu_write_ctx_desc() to modify context descriptor table entries. To keep things simple we only implement one level of context descriptor tables here, but as with stream and page tables, an SSID can be split to index multiple levels of tables. Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Signed-off-by: Will Deacon <will@kernel.org>
2020-01-15iommu/arm-smmu-v3: Add context descriptor tables allocatorsJean-Philippe Brucker1-13/+34
Support for SSID will require allocating context descriptor tables. Move the context descriptor allocation to separate functions. Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Signed-off-by: Will Deacon <will@kernel.org>
2020-01-15iommu/arm-smmu-v3: Prepare arm_smmu_s1_cfg for SSID supportJean-Philippe Brucker1-19/+27
When adding SSID support to the SMMUv3 driver, we'll need to manipulate leaf pasid tables and context descriptors. Extract the context descriptor structure and align with the way stream tables are handled. Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Signed-off-by: Will Deacon <will@kernel.org>
2020-01-15iommu/arm-smmu-v3: Parse PASID devicetree property of platform devicesJean-Philippe Brucker2-1/+18
For platform devices that support SubstreamID (SSID), firmware provides the number of supported SSID bits. Restrict it to what the SMMU supports and cache it into master->ssid_bits, which will also be used for PCI PASID. Reviewed-by: Eric Auger <eric.auger@redhat.com> Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Signed-off-by: Will Deacon <will@kernel.org>
2020-01-15iommu/arm-smmu-v3: Drop __GFP_ZERO flag from DMA allocationJean-Philippe Brucker1-5/+4
Since commit 518a2f1925c3 ("dma-mapping: zero memory returned from dma_alloc_*"), dma_alloc_* always initializes memory to zero, so there is no need to use dma_zalloc_* or pass the __GFP_ZERO flag anymore. The flag was introduced by commit 04fa26c71be5 ("iommu/arm-smmu: Convert DMA buffer allocations to the managed API"), since the managed API didn't provide a dmam_zalloc_coherent() function. Reviewed-by: Eric Auger <eric.auger@redhat.com> Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Signed-off-by: Will Deacon <will@kernel.org>
2020-01-10iommu/arm-smmu: Improve SMR mask testRobin Murphy1-5/+18
Make the SMR mask test more robust against SMR0 being live at probe time, which might happen once we start supporting firmware reservations for framebuffers and suchlike. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will@kernel.org>
2020-01-10iommu/io-pgtable-arm: Prepare for TTBR1 usageRobin Murphy1-6/+19
Now that we can correctly extract top-level indices without relying on the remaining upper bits being zero, the only remaining impediments to using a given table for TTBR1 are the address validation on map/unmap and the awkward TCR translation granule format. Add a quirk so that we can do the right thing at those points. Tested-by: Jordan Crouse <jcrouse@codeaurora.org> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will@kernel.org>
2020-01-10iommu/io-pgtable-arm: Rationalise VTCR handlingWill Deacon4-38/+59
Commit 05a648cd2dd7 ("iommu/io-pgtable-arm: Rationalise TCR handling") reworked the way in which the TCR register value is returned from the io-pgtable code when targetting the Arm long-descriptor format, in preparation for allowing page-tables to target TTBR1. As it turns out, the new interface is a lot nicer to use, so do the same conversion for the VTCR register even though there is only a single base register for stage-2 translation. Cc: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will@kernel.org>
2020-01-10iommu/arm-smmu: Rename public #defines under ARM_SMMU_ namespaceWill Deacon4-188/+204
Now that we have arm-smmu.h defining various SMMU constants, ensure that they are namespaced with the ARM_SMMU_ prefix in order to avoid conflicts with the CPU, such as the one we're currently bodging around with the TCR. Cc: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will@kernel.org>
2020-01-10iommu/io-pgtable-arm: Rationalise TCR handlingRobin Murphy7-103/+87
Although it's conceptually nice for the io_pgtable_cfg to provide a standard VMSA TCR value, the reality is that no VMSA-compliant IOMMU looks exactly like an Arm CPU, and they all have various other TCR controls which io-pgtable can't be expected to understand. Thus since there is an expectation that drivers will have to add to the given TCR value anyway, let's strip it down to just the essentials that are directly relevant to io-pgtable's inner workings - namely the various sizes and the walk attributes. Tested-by: Jordan Crouse <jcrouse@codeaurora.org> Signed-off-by: Robin Murphy <robin.murphy@arm.com> [will: Add missing include of bitfield.h] Signed-off-by: Will Deacon <will@kernel.org>
2020-01-10iommu/io-pgtable-arm: Ensure ARM_64_LPAE_S2_TCR_RES1 is unsignedWill Deacon1-1/+1
ARM_64_LPAE_S2_TCR_RES1 is intended to map to bit 31 of the VTCR register, which is required to be set to 1 by the architecture. Unfortunately, we accidentally treat this as a signed quantity which means we also set the upper 32 bits of the VTCR to one, and they are required to be zero. Treat ARM_64_LPAE_S2_TCR_RES1 as unsigned to avoid the unwanted sign-extension up to 64 bits. Cc: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will@kernel.org>
2020-01-10iommu/io-pgtable-arm: Improve attribute handlingRobin Murphy2-10/+14
By VMSA rules, using Normal Non-Cacheable type with a shareability attribute of anything other than Outer Shareable is liable to lead into unpredictable territory: | Overlaying the shareability attribute (B3-1377, ARM DDI 0406C.c) | | A memory region with a resultant memory type attribute of Normal, and | a resultant cacheability attribute of Inner Non-cacheable, Outer | Non-cacheable, must have a resultant shareability attribute of Outer | Shareable, otherwise shareability is UNPREDICTABLE Although the SMMU architectures seem to give some slightly stronger guarantees of Non-Cacheable output types becoming implicitly Outer Shareable in most cases, we may as well be explicit and not take any chances. It's also weird that LPAE attribute handling is currently split between prot_to_pte() and init_pte() given that it can all be statically determined up-front. Thus, collect *all* the LPAE attributes into prot_to_pte() in order to logically pick the shareability based on the incoming IOMMU API prot value, and tweak the short-descriptor code to stop setting TTBR0.NOS for Non-Cacheable walks. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will@kernel.org>
2020-01-10iommu/io-pgtable-arm: Support non-coherent stage-2 page tablesWill Deacon1-4/+10
Commit 9e6ea59f3ff3 ("iommu/io-pgtable: Support non-coherent page tables") added support for non-coherent page-table walks to the Arm IOMMU page-table backends. Unfortunately, it left the stage-2 allocator unchanged, so let's hook that up in the same way. Cc: Bjorn Andersson <bjorn.andersson@linaro.org> Cc: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will@kernel.org>
2020-01-10iommu/io-pgtable-arm: Rationalise TTBRn handlingRobin Murphy8-25/+21
TTBR1 values have so far been redundant since no users implement any support for split address spaces. Crucially, though, one of the main reasons for wanting to do so is to be able to manage each half entirely independently, e.g. context-switching one set of mappings without disturbing the other. Thus it seems unlikely that tying two tables together in a single io_pgtable_cfg would ever be particularly desirable or useful. Streamline the configs to just a single conceptual TTBR value representing the allocated table. This paves the way for future users to support split address spaces by simply allocating a table and dealing with the detailed TTBRn logistics themselves. Tested-by: Jordan Crouse <jcrouse@codeaurora.org> Signed-off-by: Robin Murphy <robin.murphy@arm.com> [will: Drop change to ttbr value] Signed-off-by: Will Deacon <will@kernel.org>
2020-01-10iommu/arm-smmu: Fix -Wunused-const-variable warningMasahiro Yamada1-1/+1
For ARCH=arm builds, OF is not necessarily enabled, that is, you can build this driver without CONFIG_OF. When CONFIG_OF is unset, of_match_ptr() is NULL, and arm_smmu_of_match is left orphan. Building it with W=1 emits a warning: drivers/iommu/arm-smmu.c:1904:34: warning: ‘arm_smmu_of_match’ defined but not used [-Wunused-const-variable=] static const struct of_device_id arm_smmu_of_match[] = { ^~~~~~~~~~~~~~~~~ There are two ways to fix this: - annotate arm_smmu_of_match with __maybe_unused (or surround the code with #ifdef CONFIG_OF ... #endif) - stop using of_match_ptr() This commit took the latter solution. It slightly increases the object size, but it is probably not a big deal because arm_smmu_device_dt_probe() is also compiled irrespective of CONFIG_OF. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Signed-off-by: Will Deacon <will@kernel.org>
2020-01-10iommu/arm-smmu-v3: Remove useless of_match_ptr()Masahiro Yamada1-1/+1
The CONFIG option controlling this driver, ARM_SMMU_V3, depends on ARM64, which select's OF. So, CONFIG_OF is always defined when building this driver. of_match_ptr(arm_smmu_of_match) is the same as arm_smmu_of_match. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Signed-off-by: Will Deacon <will@kernel.org>
2020-01-10iommu/arm-smmu-v3: Fix resource_size checkMasahiro Yamada1-1/+1
This is an off-by-one mistake. resource_size() returns res->end - res->start + 1. Reviewed-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Signed-off-by: Will Deacon <will@kernel.org>
2020-01-10iommu/arm-smmu-v3: Populate VMID field for CMDQ_OP_TLBI_NH_VAShameer Kolothum1-0/+1
CMDQ_OP_TLBI_NH_VA requires VMID and this was missing since commit 1c27df1c0a82 ("iommu/arm-smmu: Use correct address mask for CMD_TLBI_S2_IPA"). Add it back. Fixes: 1c27df1c0a82 ("iommu/arm-smmu: Use correct address mask for CMD_TLBI_S2_IPA") Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com> Signed-off-by: Will Deacon <will@kernel.org>
2020-01-10drivers/iommu: Initialise module 'owner' field in iommu_device_set_ops()Will Deacon2-2/+0
Requiring each IOMMU driver to initialise the 'owner' field of their 'struct iommu_ops' is error-prone and easily forgotten. Follow the example set by PCI and USB by assigning THIS_MODULE automatically when registering the ops structure with IOMMU core. Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Suggested-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Signed-off-by: Will Deacon <will@kernel.org>
2020-01-07iommu/dma: fix variable 'cookie' set but not usedQian Cai1-3/+0
The commit c18647900ec8 ("iommu/dma: Relax locking in iommu_dma_prepare_msi()") introduced a compliation warning, drivers/iommu/dma-iommu.c: In function 'iommu_dma_prepare_msi': drivers/iommu/dma-iommu.c:1206:27: warning: variable 'cookie' set but not used [-Wunused-but-set-variable] struct iommu_dma_cookie *cookie; ^~~~~~ Fixes: c18647900ec8 ("iommu/dma: Relax locking in iommu_dma_prepare_msi()") Signed-off-by: Qian Cai <cai@lca.pw> Acked-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-01-07iommu/vt-d: Unlink device if failed to add to groupJon Derrick1-3/+10
If the device fails to be added to the group, make sure to unlink the reference before returning. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Fixes: 39ab9555c2411 ("iommu: Add sysfs bindings for struct iommu_device") Acked-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-01-07iommu: Remove device link to group on failureJon Derrick1-0/+1
This adds the missing teardown step that removes the device link from the group when the device addition fails. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Fixes: 797a8b4d768c5 ("iommu: Handle default domain attach failure") Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-01-07iommu/vt-d: Fix adding non-PCI devices to Intel IOMMUPatrick Steinhardt1-1/+8
Starting with commit fa212a97f3a3 ("iommu/vt-d: Probe DMA-capable ACPI name space devices"), we now probe DMA-capable ACPI name space devices. On Dell XPS 13 9343, which has an Intel LPSS platform device INTL9C60 enumerated via ACPI, this change leads to the following warning: ------------[ cut here ]------------ WARNING: CPU: 1 PID: 1 at pci_device_group+0x11a/0x130 CPU: 1 PID: 1 Comm: swapper/0 Tainted: G T 5.5.0-rc3+ #22 Hardware name: Dell Inc. XPS 13 9343/0310JH, BIOS A20 06/06/2019 RIP: 0010:pci_device_group+0x11a/0x130 Code: f0 ff ff 48 85 c0 49 89 c4 75 c4 48 8d 74 24 10 48 89 ef e8 48 ef ff ff 48 85 c0 49 89 c4 75 af e8 db f7 ff ff 49 89 c4 eb a5 <0f> 0b 49 c7 c4 ea ff ff ff eb 9a e8 96 1e c7 ff 66 0f 1f 44 00 00 RSP: 0000:ffffc0d6c0043cb0 EFLAGS: 00010202 RAX: 0000000000000000 RBX: ffffa3d1d43dd810 RCX: 0000000000000000 RDX: ffffa3d1d4fecf80 RSI: ffffa3d12943dcc0 RDI: ffffa3d1d43dd810 RBP: ffffa3d1d43dd810 R08: 0000000000000000 R09: ffffa3d1d4c04a80 R10: ffffa3d1d4c00880 R11: ffffa3d1d44ba000 R12: 0000000000000000 R13: ffffa3d1d4383b80 R14: ffffa3d1d4c090d0 R15: ffffa3d1d4324530 FS: 0000000000000000(0000) GS:ffffa3d1d6700000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 000000000460a001 CR4: 00000000003606e0 Call Trace: ? iommu_group_get_for_dev+0x81/0x1f0 ? intel_iommu_add_device+0x61/0x170 ? iommu_probe_device+0x43/0xd0 ? intel_iommu_init+0x1fa2/0x2235 ? pci_iommu_init+0x52/0xe7 ? e820__memblock_setup+0x15c/0x15c ? do_one_initcall+0xcc/0x27e ? kernel_init_freeable+0x169/0x259 ? rest_init+0x95/0x95 ? kernel_init+0x5/0xeb ? ret_from_fork+0x35/0x40 ---[ end trace 28473e7abc25b92c ]--- DMAR: ACPI name space devices didn't probe correctly The bug results from the fact that while we now enumerate ACPI devices, we aren't able to handle any non-PCI device when generating the device group. Fix the issue by implementing an Intel-specific callback that returns `pci_device_group` only if the device is a PCI device. Otherwise, it will return a generic device group. Fixes: fa212a97f3a3 ("iommu/vt-d: Probe DMA-capable ACPI name space devices") Signed-off-by: Patrick Steinhardt <ps@pks.im> Cc: stable@vger.kernel.org # v5.3+ Acked-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-01-07iommu/amd: Fix typos for PPR macrosAdrian Huang2-4/+4
The bit 13 and bit 14 of the IOMMU control register are PPRLogEn and PPRIntEn. They are related to PPR (Peripheral Page Request) instead of 'PPF'. Fix them accrodingly. Signed-off-by: Adrian Huang <ahuang12@lenovo.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-01-07iommu/amd: Remove local variablesAdrian Huang1-5/+1
The usage of the local variables 'range' and 'misc' was removed from commit 226e889b20a9 ("iommu/amd: Remove first/last_device handling") and commit 23c742db2171 ("iommu/amd: Split out PCI related parts of IOMMU initialization"). So, remove them accrodingly. Signed-off-by: Adrian Huang <ahuang12@lenovo.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-01-07iommu/vt-d: debugfs: Add support to show page table internalsLu Baolu2-2/+77
Export page table internals of the domain attached to each device. Example of such dump on a Skylake machine: $ sudo cat /sys/kernel/debug/iommu/intel/domain_translation_struct [ ... ] Device 0000:00:14.0 with pasid 0 @0x15f3d9000 IOVA_PFN PML5E PML4E 0x000000008ced0 | 0x0000000000000000 0x000000015f3da003 0x000000008ced1 | 0x0000000000000000 0x000000015f3da003 0x000000008ced2 | 0x0000000000000000 0x000000015f3da003 0x000000008ced3 | 0x0000000000000000 0x000000015f3da003 0x000000008ced4 | 0x0000000000000000 0x000000015f3da003 0x000000008ced5 | 0x0000000000000000 0x000000015f3da003 0x000000008ced6 | 0x0000000000000000 0x000000015f3da003 0x000000008ced7 | 0x0000000000000000 0x000000015f3da003 0x000000008ced8 | 0x0000000000000000 0x000000015f3da003 0x000000008ced9 | 0x0000000000000000 0x000000015f3da003 PDPE PDE PTE 0x000000015f3db003 0x000000015f3dc003 0x000000008ced0003 0x000000015f3db003 0x000000015f3dc003 0x000000008ced1003 0x000000015f3db003 0x000000015f3dc003 0x000000008ced2003 0x000000015f3db003 0x000000015f3dc003 0x000000008ced3003 0x000000015f3db003 0x000000015f3dc003 0x000000008ced4003 0x000000015f3db003 0x000000015f3dc003 0x000000008ced5003 0x000000015f3db003 0x000000015f3dc003 0x000000008ced6003 0x000000015f3db003 0x000000015f3dc003 0x000000008ced7003 0x000000015f3db003 0x000000015f3dc003 0x000000008ced8003 0x000000015f3db003 0x000000015f3dc003 0x000000008ced9003 [ ... ] Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
Copyright © 1996 – 2023 Jason A. Donenfeld. All Rights Reverse Engineered.