aboutsummaryrefslogtreecommitdiffstats
path: root/drivers/iommu (follow)
AgeCommit message (Collapse)AuthorFilesLines
2018-11-22iommu/vt-d: Use memunmap to free memremapPan Bian1-1/+1
memunmap() should be used to free the return of memremap(), not iounmap(). Fixes: dfddb969edf0 ('iommu/vt-d: Switch from ioremap_cache to memremap') Signed-off-by: Pan Bian <bianpan2016@163.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-11-12amd/iommu: Fix Guest Virtual APIC Log Tail Address RegisterFilippo Sironi1-1/+2
This register should have been programmed with the physical address of the memory location containing the shadow tail pointer for the guest virtual APIC log instead of the base address. Fixes: 8bda0cfbdc1a ('iommu/amd: Detect and initialize guest vAPIC log') Signed-off-by: Filippo Sironi <sironi@amazon.de> Signed-off-by: Wei Wang <wawei@amazon.de> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-11-08iommu/ipmmu-vmsa: Fix crash on early domain freeGeert Uytterhoeven1-0/+3
If iommu_ops.add_device() fails, iommu_ops.domain_free() is still called, leading to a crash, as the domain was only partially initialized: ipmmu-vmsa e67b0000.mmu: Cannot accommodate DMA translation for IOMMU page tables sata_rcar ee300000.sata: Unable to initialize IPMMU context iommu: Failed to add device ee300000.sata to group 0: -22 Unable to handle kernel NULL pointer dereference at virtual address 0000000000000038 ... Call trace: ipmmu_domain_free+0x1c/0xa0 iommu_group_release+0x48/0x68 kobject_put+0x74/0xe8 kobject_del.part.0+0x3c/0x50 kobject_put+0x60/0xe8 iommu_group_get_for_dev+0xa8/0x1f0 ipmmu_add_device+0x1c/0x40 of_iommu_configure+0x118/0x190 Fix this by checking if the domain's context already exists, before trying to destroy it. Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be> Reviewed-by: Robin Murphy <robin.murphy@arm.com> Fixes: d25a2a16f0889 ('iommu: Add driver for Renesas VMSA-compatible IPMMU') Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-11-06iommu/vt-d: Fix NULL pointer dereference in prq_event_thread()Lu Baolu1-1/+1
When handling page request without pasid event, go to "no_pasid" branch instead of "bad_req". Otherwise, a NULL pointer deference will happen there. Cc: Ashok Raj <ashok.raj@intel.com> Cc: Jacob Pan <jacob.jun.pan@linux.intel.com> Cc: Sohil Mehta <sohil.mehta@intel.com> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Fixes: a222a7f0bb6c9 'iommu/vt-d: Implement page request handling' Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-10-31mm: remove include/linux/bootmem.hMike Rapoport2-2/+2
Move remaining definitions and declarations from include/linux/bootmem.h into include/linux/memblock.h and remove the redundant header. The includes were replaced with the semantic patch below and then semi-automated removal of duplicated '#include <linux/memblock.h> @@ @@ - #include <linux/bootmem.h> + #include <linux/memblock.h> [sfr@canb.auug.org.au: dma-direct: fix up for the removal of linux/bootmem.h] Link: http://lkml.kernel.org/r/20181002185342.133d1680@canb.auug.org.au [sfr@canb.auug.org.au: powerpc: fix up for removal of linux/bootmem.h] Link: http://lkml.kernel.org/r/20181005161406.73ef8727@canb.auug.org.au [sfr@canb.auug.org.au: x86/kaslr, ACPI/NUMA: fix for linux/bootmem.h removal] Link: http://lkml.kernel.org/r/20181008190341.5e396491@canb.auug.org.au Link: http://lkml.kernel.org/r/1536927045-23536-30-git-send-email-rppt@linux.vnet.ibm.com Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com> Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au> Acked-by: Michal Hocko <mhocko@suse.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Chris Zankel <chris@zankel.net> Cc: "David S. Miller" <davem@davemloft.net> Cc: Geert Uytterhoeven <geert@linux-m68k.org> Cc: Greentime Hu <green.hu@gmail.com> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: Guan Xuetao <gxt@pku.edu.cn> Cc: Ingo Molnar <mingo@redhat.com> Cc: "James E.J. Bottomley" <jejb@parisc-linux.org> Cc: Jonas Bonn <jonas@southpole.se> Cc: Jonathan Corbet <corbet@lwn.net> Cc: Ley Foon Tan <lftan@altera.com> Cc: Mark Salter <msalter@redhat.com> Cc: Martin Schwidefsky <schwidefsky@de.ibm.com> Cc: Matt Turner <mattst88@gmail.com> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Michal Simek <monstr@monstr.eu> Cc: Palmer Dabbelt <palmer@sifive.com> Cc: Paul Burton <paul.burton@mips.com> Cc: Richard Kuo <rkuo@codeaurora.org> Cc: Richard Weinberger <richard@nod.at> Cc: Rich Felker <dalias@libc.org> Cc: Russell King <linux@armlinux.org.uk> Cc: Serge Semin <fancer.lancer@gmail.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Tony Luck <tony.luck@intel.com> Cc: Vineet Gupta <vgupta@synopsys.com> Cc: Yoshinori Sato <ysato@users.sourceforge.jp> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2018-10-26Merge branch 'akpm' (patches from Andrew)Linus Torvalds2-2/+0
Merge updates from Andrew Morton: - a few misc things - ocfs2 updates - most of MM * emailed patches from Andrew Morton <akpm@linux-foundation.org>: (132 commits) hugetlbfs: dirty pages as they are added to pagecache mm: export add_swap_extent() mm: split SWP_FILE into SWP_ACTIVATED and SWP_FS tools/testing/selftests/vm/map_fixed_noreplace.c: add test for MAP_FIXED_NOREPLACE mm: thp: relocate flush_cache_range() in migrate_misplaced_transhuge_page() mm: thp: fix mmu_notifier in migrate_misplaced_transhuge_page() mm: thp: fix MADV_DONTNEED vs migrate_misplaced_transhuge_page race condition mm/kasan/quarantine.c: make quarantine_lock a raw_spinlock_t mm/gup: cache dev_pagemap while pinning pages Revert "x86/e820: put !E820_TYPE_RAM regions into memblock.reserved" mm: return zero_resv_unavail optimization mm: zero remaining unavailable struct pages tools/testing/selftests/vm/gup_benchmark.c: add MAP_HUGETLB option tools/testing/selftests/vm/gup_benchmark.c: add MAP_SHARED option tools/testing/selftests/vm/gup_benchmark.c: allow user specified file tools/testing/selftests/vm/gup_benchmark.c: fix 'write' flag usage mm/gup_benchmark.c: add additional pinning methods mm/gup_benchmark.c: time put_page() mm: don't raise MEMCG_OOM event due to failed high-order allocation mm/page-writeback.c: fix range_cyclic writeback vs writepages deadlock ...
2018-10-26Revert "mm, mmu_notifier: annotate mmu notifiers with blockable invalidate callbacks"Michal Hocko2-2/+0
Revert 5ff7091f5a2ca ("mm, mmu_notifier: annotate mmu notifiers with blockable invalidate callbacks"). MMU_INVALIDATE_DOES_NOT_BLOCK flags was the only one used and it is no longer needed since 93065ac753e4 ("mm, oom: distinguish blockable mode for mmu notifiers"). We now have a full support for per range !blocking behavior so we can drop the stop gap workaround which the per notifier flag was used for. Link: http://lkml.kernel.org/r/20180827112623.8992-4-mhocko@kernel.org Signed-off-by: Michal Hocko <mhocko@suse.com> Cc: David Rientjes <rientjes@google.com> Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com> Cc: Jerome Glisse <jglisse@redhat.com> Cc: Juergen Gross <jgross@suse.com> Cc: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2018-10-26Merge tag 'devicetree-for-4.20' of git://git.kernel.org/pub/scm/linux/kernel/git/robh/linuxLinus Torvalds1-1/+1
Pull Devicetree updates from Rob Herring: "A bit bigger than normal as I've been busy this cycle. There's a few things with dependencies and a few things subsystem maintainers didn't pick up, so I'm taking them thru my tree. The fixes from Johan didn't get into linux-next, but they've been waiting for some time now and they are what's left of what subsystem maintainers didn't pick up. Summary: - Sync dtc with upstream version v1.4.7-14-gc86da84d30e4 - Work to get rid of direct accesses to struct device_node name and type pointers in preparation for removing them. New helpers for parsing DT cpu nodes and conversions to use the helpers. printk conversions to %pOFn for printing DT node names. Most went thru subystem trees, so this is the remainder. - Fixes to DT child node lookups to actually be restricted to child nodes instead of treewide. - Refactoring of dtb targets out of arch code. This makes the support more uniform and enables building all dtbs on c6x, microblaze, and powerpc. - Various DT binding updates for Renesas r8a7744 SoC - Vendor prefixes for Facebook, OLPC - Restructuring of some ARM binding docs moving some peripheral bindings out of board/SoC binding files - New "secure-chosen" binding for secure world settings on ARM - Dual licensing of 2 DT IRQ binding headers" * tag 'devicetree-for-4.20' of git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux: (78 commits) ARM: dt: relicense two DT binding IRQ headers power: supply: twl4030-charger: fix OF sibling-node lookup NFC: nfcmrvl_uart: fix OF child-node lookup net: stmmac: dwmac-sun8i: fix OF child-node lookup net: bcmgenet: fix OF child-node lookup drm/msm: fix OF child-node lookup drm/mediatek: fix OF sibling-node lookup of: Add missing exports of node name compare functions dt-bindings: Add OLPC vendor prefix dt-bindings: misc: bk4: Add device tree binding for Liebherr's BK4 SPI bus dt-bindings: thermal: samsung: Add SPDX license identifier dt-bindings: clock: samsung: Add SPDX license identifiers dt-bindings: timer: ostm: Add R7S9210 support dt-bindings: phy: rcar-gen2: Add r8a7744 support dt-bindings: can: rcar_can: Add r8a7744 support dt-bindings: timer: renesas, cmt: Document r8a7744 CMT support dt-bindings: watchdog: renesas-wdt: Document r8a7744 support dt-bindings: thermal: rcar: Add device tree support for r8a7744 Documentation: dt: Add binding for /secure-chosen/stdout-path dt-bindings: arm: zte: Move sysctrl bindings to their own doc ...
2018-10-26Merge tag 'iommu-updates-v4.20' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommuLinus Torvalds18-226/+709
Pull IOMMU updates from Joerg Roedel: - Debugfs support for the Intel VT-d driver. When enabled, it now also exposes some of its internal data structures to user-space for debugging purposes. - ARM-SMMU driver now uses the generic deferred flushing and fast-path iova allocation code. This is expected to be a major performance improvement, as this allocation path scales a lot better. - Support for r8a7744 in the Renesas iommu driver - Couple of minor fixes and improvements all over the place * tag 'iommu-updates-v4.20' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: (39 commits) iommu/arm-smmu-v3: Remove unnecessary wrapper function iommu/arm-smmu-v3: Add SPDX header iommu/amd: Add default branch in amd_iommu_capable() dt-bindings: iommu: ipmmu-vmsa: Add r8a7744 support iommu/amd: Move iommu_init_pci() to .init section iommu/arm-smmu: Support non-strict mode iommu/io-pgtable-arm-v7s: Add support for non-strict mode iommu/arm-smmu-v3: Add support for non-strict mode iommu/io-pgtable-arm: Add support for non-strict mode iommu: Add "iommu.strict" command line option iommu/dma: Add support for non-strict mode iommu/arm-smmu: Ensure that page-table updates are visible before TLBI iommu/arm-smmu-v3: Implement flush_iotlb_all hook iommu/arm-smmu-v3: Avoid back-to-back CMD_SYNC operations iommu/arm-smmu-v3: Fix unexpected CMD_SYNC timeout iommu/io-pgtable-arm: Fix race handling in split_blk_unmap() iommu/arm-smmu-v3: Fix a couple of minor comment typos iommu: Fix a typo iommu: Remove .domain_{get,set}_windows iommu: Tidy up window attributes ...
2018-10-25Merge tag 'kvm-4.20-1' of git://git.kernel.org/pub/scm/virt/kvm/kvmLinus Torvalds1-0/+8
Pull KVM updates from Radim Krčmář: "ARM: - Improved guest IPA space support (32 to 52 bits) - RAS event delivery for 32bit - PMU fixes - Guest entry hardening - Various cleanups - Port of dirty_log_test selftest PPC: - Nested HV KVM support for radix guests on POWER9. The performance is much better than with PR KVM. Migration and arbitrary level of nesting is supported. - Disable nested HV-KVM on early POWER9 chips that need a particular hardware bug workaround - One VM per core mode to prevent potential data leaks - PCI pass-through optimization - merge ppc-kvm topic branch and kvm-ppc-fixes to get a better base s390: - Initial version of AP crypto virtualization via vfio-mdev - Improvement for vfio-ap - Set the host program identifier - Optimize page table locking x86: - Enable nested virtualization by default - Implement Hyper-V IPI hypercalls - Improve #PF and #DB handling - Allow guests to use Enlightened VMCS - Add migration selftests for VMCS and Enlightened VMCS - Allow coalesced PIO accesses - Add an option to perform nested VMCS host state consistency check through hardware - Automatic tuning of lapic_timer_advance_ns - Many fixes, minor improvements, and cleanups" * tag 'kvm-4.20-1' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (204 commits) KVM/nVMX: Do not validate that posted_intr_desc_addr is page aligned Revert "kvm: x86: optimize dr6 restore" KVM: PPC: Optimize clearing TCEs for sparse tables x86/kvm/nVMX: tweak shadow fields selftests/kvm: add missing executables to .gitignore KVM: arm64: Safety check PSTATE when entering guest and handle IL KVM: PPC: Book3S HV: Don't use streamlined entry path on early POWER9 chips arm/arm64: KVM: Enable 32 bits kvm vcpu events support arm/arm64: KVM: Rename function kvm_arch_dev_ioctl_check_extension() KVM: arm64: Fix caching of host MDCR_EL2 value KVM: VMX: enable nested virtualization by default KVM/x86: Use 32bit xor to clear registers in svm.c kvm: x86: Introduce KVM_CAP_EXCEPTION_PAYLOAD kvm: vmx: Defer setting of DR6 until #DB delivery kvm: x86: Defer setting of CR2 until #PF delivery kvm: x86: Add payload operands to kvm_multiple_exception kvm: x86: Add exception payload fields to kvm_vcpu_events kvm: x86: Add has_payload and payload to kvm_queued_exception KVM: Documentation: Fix omission in struct kvm_vcpu_events KVM: selftests: add Enlightened VMCS test ...
2018-10-23Merge branch 'x86-mm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tipLinus Torvalds1-2/+12
Pull x86 mm updates from Ingo Molnar: "Lots of changes in this cycle: - Lots of CPA (change page attribute) optimizations and related cleanups (Thomas Gleixner, Peter Zijstra) - Make lazy TLB mode even lazier (Rik van Riel) - Fault handler cleanups and improvements (Dave Hansen) - kdump, vmcore: Enable kdumping encrypted memory with AMD SME enabled (Lianbo Jiang) - Clean up VM layout documentation (Baoquan He, Ingo Molnar) - ... plus misc other fixes and enhancements" * 'x86-mm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (51 commits) x86/stackprotector: Remove the call to boot_init_stack_canary() from cpu_startup_entry() x86/mm: Kill stray kernel fault handling comment x86/mm: Do not warn about PCI BIOS W+X mappings resource: Clean it up a bit resource: Fix find_next_iomem_res() iteration issue resource: Include resource end in walk_*() interfaces x86/kexec: Correct KEXEC_BACKUP_SRC_END off-by-one error x86/mm: Remove spurious fault pkey check x86/mm/vsyscall: Consider vsyscall page part of user address space x86/mm: Add vsyscall address helper x86/mm: Fix exception table comments x86/mm: Add clarifying comments for user addr space x86/mm: Break out user address space handling x86/mm: Break out kernel address space handling x86/mm: Clarify hardware vs. software "error_code" x86/mm/tlb: Make lazy TLB mode lazier x86/mm/tlb: Add freed_tables element to flush_tlb_info x86/mm/tlb: Add freed_tables argument to flush_tlb_mm_range smp,cpumask: introduce on_each_cpu_cond_mask smp: use __cpumask_set_cpu in on_each_cpu_cond ...
2018-10-23Merge tag 'please-pull-next' of git://git.kernel.org/pub/scm/linux/kernel/git/aegl/linuxLinus Torvalds1-3/+1
Pull ia64 updates from Tony Luck: "Miscellaneous ia64 fixes from Christoph" * tag 'please-pull-next' of git://git.kernel.org/pub/scm/linux/kernel/git/aegl/linux: intel-iommu: mark intel_dma_ops static ia64: remove machvec_dma_sync_{single,sg} ia64/sn2: remove no-ops dma sync methods ia64: remove the unused iommu_dma_init function ia64: remove the unused pci_iommu_shutdown function ia64: remove the unused bad_dma_address symbol ia64: remove iommu_dma_supported ia64: remove the dead iommu_sac_force variable ia64: remove the kern_mem_attribute export
2018-10-10Merge branches 'arm/renesas', 'arm/smmu', 'ppc/pamu', 'x86/vt-d', 'x86/amd' and 'core' into nextJoerg Roedel18-226/+709
2018-10-10iommu/arm-smmu-v3: Remove unnecessary wrapper functionAndrew Murray1-8/+4
Simplify the code by removing an unnecessary wrapper function. This was left behind by commit 2f657add07a8 ("iommu/arm-smmu-v3: Specialise CMD_SYNC handling") Signed-off-by: Andrew Murray <andrew.murray@arm.com> Reviewed-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-10-10iommu/arm-smmu-v3: Add SPDX headerAndrew Murray1-12/+1
Replace license text with SDPX header Signed-off-by: Andrew Murray <andrew.murray@arm.com> Acked-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-10-08iommu/amd: Add default branch in amd_iommu_capable()Lu Baolu1-0/+2
Recent gcc warns about switching on an enumeration, but not having an explicit case statement for all members of the enumeration. To show the compiler this is intentional, we simply add a default case with nothing more than a break statement. Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-10-06iommu/amd: Remap the IOMMU device table with the memory encryption mask for kdumpLianbo Jiang1-2/+12
The kdump kernel copies the IOMMU device table from the old device table which is encrypted when SME is enabled in the first kernel. So remap the old device table with the memory encryption mask in the kdump kernel. [ bp: Massage commit message. ] Signed-off-by: Lianbo Jiang <lijiang@redhat.com> Signed-off-by: Borislav Petkov <bp@suse.de> Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com> Acked-by: Joerg Roedel <jroedel@suse.de> Cc: kexec@lists.infradead.org Cc: tglx@linutronix.de Cc: mingo@redhat.com Cc: hpa@zytor.com Cc: akpm@linux-foundation.org Cc: dan.j.williams@intel.com Cc: bhelgaas@google.com Cc: baiyaowei@cmss.chinamobile.com Cc: tiwai@suse.de Cc: brijesh.singh@amd.com Cc: dyoung@redhat.com Cc: bhe@redhat.com Link: https://lkml.kernel.org/r/20180930031033.22110-4-lijiang@redhat.com
2018-10-05iommu/amd: Move iommu_init_pci() to .init sectionJoerg Roedel1-1/+1
The function is only called from another __init function, so it should be moved to .init too. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-10-05iommu/amd: Clear memory encryption mask from physical addressSingh, Brijesh1-1/+1
Boris Ostrovsky reported a memory leak with device passthrough when SME is active. The VFIO driver uses iommu_iova_to_phys() to get the physical address for an iova. This physical address is later passed into vfio_unmap_unpin() to unpin the memory. The vfio_unmap_unpin() uses pfn_valid() before unpinning the memory. The pfn_valid() check was failing because encryption mask was part of the physical address returned. This resulted in the memory not being unpinned and therefore leaked after the guest terminates. The memory encryption mask must be cleared from the physical address in iommu_iova_to_phys(). Fixes: 2543a786aa25 ("iommu/amd: Allow the AMD IOMMU to work with memory encryption") Reported-by: Boris Ostrovsky <boris.ostrovsky@oracle.com> Cc: Tom Lendacky <thomas.lendacky@amd.com> Cc: Joerg Roedel <joro@8bytes.org> Cc: <iommu@lists.linux-foundation.org> Cc: Borislav Petkov <bp@suse.de> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Radim Krčmář <rkrcmar@redhat.com> Cc: kvm@vger.kernel.org Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com> Cc: <stable@vger.kernel.org> # 4.14+ Signed-off-by: Brijesh Singh <brijesh.singh@amd.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-10-01Merge branch 'for-joerg/arm-smmu/updates' of git://git.kernel.org/pub/scm/linux/kernel/git/will/linux into arm/smmuJoerg Roedel7-69/+230
2018-10-01iommu/arm-smmu: Support non-strict modeRobin Murphy1-27/+66
All we need is to wire up .flush_iotlb_all properly and implement the domain attribute, and iommu-dma and io-pgtable will do the rest for us. The only real subtlety is documenting the barrier semantics we're introducing between io-pgtable and the drivers for non-strict flushes. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2018-10-01iommu/io-pgtable-arm-v7s: Add support for non-strict modeRobin Murphy1-1/+10
As for LPAE, it's simply a case of skipping the leaf invalidation for a regular unmap, and ensuring that the one in split_blk_unmap() is paired with an explicit sync ASAP rather than relying on one which might only eventually happen way down the line. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2018-10-01iommu/arm-smmu-v3: Add support for non-strict modeZhen Lei1-23/+56
Now that io-pgtable knows how to dodge strict TLB maintenance, all that's left to do is bridge the gap between the IOMMU core requesting DOMAIN_ATTR_DMA_USE_FLUSH_QUEUE for default domains, and showing the appropriate IO_PGTABLE_QUIRK_NON_STRICT flag to alloc_io_pgtable_ops(). Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com> [rm: convert to domain attribute, tweak commit message] Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2018-10-01iommu/io-pgtable-arm: Add support for non-strict modeZhen Lei2-2/+17
Non-strict mode is simply a case of skipping 'regular' leaf TLBIs, since the sync is already factored out into ops->iotlb_sync at the core API level. Non-leaf invalidations where we change the page table structure itself still have to be issued synchronously in order to maintain walk caches correctly. To save having to reason about it too much, make sure the invalidation in arm_lpae_split_blk_unmap() just performs its own unconditional sync to minimise the window in which we're technically violating the break- before-make requirement on a live mapping. This might work out redundant with an outer-level sync for strict unmaps, but we'll never be splitting blocks on a DMA fastpath anyway. Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com> [rm: tweak comment, commit message, split_blk_unmap logic and barriers] Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2018-10-01iommu: Add "iommu.strict" command line optionZhen Lei1-0/+14
Add a generic command line option to enable lazy unmapping via IOVA flush queues, which will initally be suuported by iommu-dma. This echoes the semantics of "intel_iommu=strict" (albeit with the opposite default value), but in the driver-agnostic fashion of "iommu.passthrough". Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com> [rm: move handling out of SMMUv3 driver, clean up documentation] Signed-off-by: Robin Murphy <robin.murphy@arm.com> [will: dropped broken printk when parsing command-line option] Signed-off-by: Will Deacon <will.deacon@arm.com>
2018-10-01iommu/dma: Add support for non-strict modeZhen Lei1-1/+31
With the flush queue infrastructure already abstracted into IOVA domains, hooking it up in iommu-dma is pretty simple. Since there is a degree of dependency on the IOMMU driver knowing what to do to play along, we key the whole thing off a domain attribute which will be set on default DMA ops domains to request non-strict invalidation. That way, drivers can indicate the appropriate support by acknowledging the attribute, and we can easily fall back to strict invalidation otherwise. The flush queue callback needs a handle on the iommu_domain which owns our cookie, so we have to add a pointer back to that, but neatly, that's also sufficient to indicate whether we're using a flush queue or not, and thus which way to release IOVAs. The only slight subtlety is switching __iommu_dma_unmap() from calling iommu_unmap() to explicit iommu_unmap_fast()/iommu_tlb_sync() so that we can elide the sync entirely in non-strict mode. Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com> [rm: convert to domain attribute, tweak comments and commit message] Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2018-10-01iommu/arm-smmu: Ensure that page-table updates are visible before TLBIWill Deacon1-0/+6
The IO-pgtable code relies on the driver TLB invalidation callbacks to ensure that all page-table updates are visible to the IOMMU page-table walker. In the case that the page-table walker is cache-coherent, we cannot rely on an implicit DSB from the DMA-mapping code, so we must ensure that we execute a DSB in our tlb_add_flush() callback prior to triggering the invalidation. Cc: <stable@vger.kernel.org> Cc: Robin Murphy <robin.murphy@arm.com> Fixes: 2df7a25ce4a7 ("iommu/arm-smmu: Clean up DMA API usage") Signed-off-by: Will Deacon <will.deacon@arm.com>
2018-10-01iommu/arm-smmu-v3: Implement flush_iotlb_all hookZhen Lei1-1/+9
.flush_iotlb_all is currently stubbed to arm_smmu_iotlb_sync() since the only time it would ever need to actually do anything is for callers doing their own explicit batching, e.g.: iommu_unmap_fast(domain, ...); iommu_unmap_fast(domain, ...); iommu_iotlb_flush_all(domain, ...); where since io-pgtable still issues the TLBI commands implicitly in the unmap instead of implementing .iotlb_range_add, the "flush" only needs to ensure completion of those already-in-flight invalidations. However, we're about to start using it in anger with flush queues, so let's get a proper implementation wired up. Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com> Reviewed-by: Robin Murphy <robin.murphy@arm.com> [rm: document why it wasn't a bug] Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2018-10-01iommu/arm-smmu-v3: Avoid back-to-back CMD_SYNC operationsZhen Lei1-3/+13
Putting adjacent CMD_SYNCs into the command queue is nonsensical, but can happen when multiple CPUs are inserting commands. Rather than leave the poor old hardware to chew through these operations, we can instead drop the subsequent SYNCs and poll for completion of the first. This has been shown to improve IO performance under pressure, where the number of SYNC operations reduces by about a third: CMD_SYNCs reduced: 19542181 CMD_SYNCs total: 58098548 (include reduced) CMDs total: 116197099 (TLBI:SYNC about 1:1) Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2018-10-01iommu/arm-smmu-v3: Fix unexpected CMD_SYNC timeoutZhen Lei1-5/+3
The condition break condition of: (int)(VAL - sync_idx) >= 0 in the __arm_smmu_sync_poll_msi() polling loop requires that sync_idx must be increased monotonically according to the sequence of the CMDs in the cmdq. However, since the msidata is populated using atomic_inc_return_relaxed() before taking the command-queue spinlock, then the following scenario can occur: CPU0 CPU1 msidata=0 msidata=1 insert cmd1 insert cmd0 smmu execute cmd1 smmu execute cmd0 poll timeout, because msidata=1 is overridden by cmd0, that means VAL=0, sync_idx=1. This is not a functional problem, since the caller will eventually either timeout or exit due to another CMD_SYNC, however it's clearly not what the code is supposed to be doing. Fix it, by incrementing the sequence count with the command-queue lock held, allowing us to drop the atomic operations altogether. Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com> [will: dropped the specialised cmd building routine for now] Signed-off-by: Will Deacon <will.deacon@arm.com>
2018-10-01iommu/io-pgtable-arm: Fix race handling in split_blk_unmap()Robin Murphy1-5/+4
In removing the pagetable-wide lock, we gained the possibility of the vanishingly unlikely case where we have a race between two concurrent unmappers splitting the same block entry. The logic to handle this is fairly straightforward - whoever loses the race frees their partial next-level table and instead dereferences the winner's newly-installed entry in order to fall back to a regular unmap, which intentionally echoes the pre-existing case of recursively splitting a 1GB block down to 4KB pages by installing a full table of 2MB blocks first. Unfortunately, the chump who implemented that logic failed to update the condition check for that fallback, meaning that if said race occurs at the last level (where the loser's unmap_idx is valid) then the unmap won't actually happen. Fix that to properly account for both the race and recursive cases. Fixes: 2c3d273eabe8 ("iommu/io-pgtable-arm: Support lockless operation") Signed-off-by: Robin Murphy <robin.murphy@arm.com> [will: re-jig control flow to avoid duplicate cmpxchg test] Signed-off-by: Will Deacon <will.deacon@arm.com>
2018-10-01iommu/arm-smmu-v3: Fix a couple of minor comment typosJohn Garry1-3/+3
Fix some comment typos spotted. Signed-off-by: John Garry <john.garry@huawei.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2018-09-28iommu: fsl_pamu: use for_each_of_cpu_node iteratorRob Herring1-1/+1
Use the for_each_of_cpu_node iterator to iterate over cpu nodes. This has the side effect of defaulting to iterating using "cpu" node names in preference to the deprecated (for FDT) device_type == "cpu". Cc: Joerg Roedel <joro@8bytes.org> Cc: iommu@lists.linux-foundation.org Signed-off-by: Rob Herring <robh@kernel.org>
2018-09-26s390: vfio-ap: base implementation of VFIO AP device driverTony Krowiak1-0/+8
Introduces a new AP device driver. This device driver is built on the VFIO mediated device framework. The framework provides sysfs interfaces that facilitate passthrough access by guests to devices installed on the linux host. The VFIO AP device driver will serve two purposes: 1. Provide the interfaces to reserve AP devices for exclusive use by KVM guests. This is accomplished by unbinding the devices to be reserved for guest usage from the zcrypt device driver and binding them to the VFIO AP device driver. 2. Implements the functions, callbacks and sysfs attribute interfaces required to create one or more VFIO mediated devices each of which will be used to configure the AP matrix for a guest and serve as a file descriptor for facilitating communication between QEMU and the VFIO AP device driver. When the VFIO AP device driver is initialized: * It registers with the AP bus for control of type 10 (CEX4 and newer) AP queue devices. This limitation was imposed due to: 1. A desire to keep the code as simple as possible; 2. Some older models are no longer supported by the kernel and others are getting close to end of service. 3. A lack of older systems on which to test older devices. The probe and remove callbacks will be provided to support the binding/unbinding of AP queue devices to/from the VFIO AP device driver. * Creates a matrix device, /sys/devices/vfio_ap/matrix, to serve as the parent of the mediated devices created, one for each guest, and to hold the APQNs of the AP devices bound to the VFIO AP device driver. Signed-off-by: Tony Krowiak <akrowiak@linux.ibm.com> Reviewed-by: Halil Pasic <pasic@linux.ibm.com> Tested-by: Michael Mueller <mimu@linux.ibm.com> Tested-by: Farhan Ali <alifm@linux.ibm.com> Acked-by: David Hildenbrand <david@redhat.com> Reviewed-by: Cornelia Huck <cohuck@redhat.com> Message-Id: <20180925231641.4954-5-akrowiak@linux.vnet.ibm.com> Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
2018-09-26iommu/amd: Return devid as alias for ACPI HID devicesArindam Nath1-0/+6
ACPI HID devices do not actually have an alias for them in the IVRS. But dev_data->alias is still used for indexing into the IOMMU device table for devices being handled by the IOMMU. So for ACPI HID devices, we simply return the corresponding devid as an alias, as parsed from IVRS table. Signed-off-by: Arindam Nath <arindam.nath@amd.com> Fixes: 2bf9a0a12749 ('iommu/amd: Add iommu support for ACPI HID devices') Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-09-25iommu: Fix a typoRami Rosen1-1/+1
This patch fixes a typo in iommu.c. Signed-off-by: Rami Rosen <ramirose@gmail.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-09-25iommu: Remove .domain_{get,set}_windowsRobin Murphy1-74/+51
Since these are trivially handled by the .domain_{get,set}_attr callbacks when relevant, we can streamline struct iommu_ops for everyone. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-09-25iommu: Tidy up window attributesRobin Murphy2-20/+20
The external interface to get/set window attributes is already abstracted behind iommu_domain_{get,set}_attr(), so there's no real reason for the internal interface to be different. Since we only have one window-based driver anyway, clean up the core code by just moving the DOMAIN_ATTR_WINDOWS handling directly into the PAMU driver. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-09-25iommu/vt-d: Add debugfs support to show context internalsSohil Mehta1-0/+65
Export Intel IOMMU DMA remapping internal states, such as root and context table entries in debugfs. Example of such dump in Kabylake: /sys/kernel/debug/iommu/intel/dmar_translation_struct IOMMU dmar0: Root Table Address:45699c000 Root Table Entries: Bus: 0 H: 0 L: 45699f001 Context Table Entries for Bus: 0 Entry B:D.F High Low 16 00:02.0 102 45699e005 IOMMU dmar1: Root Table Address:45699d000 Root Table Entries: Bus: 0 H: 0 L: 45644f001 Context Table Entries for Bus: 0 Entry B:D.F High Low 160 00:14.0 102 45644e001 184 00:17.0 302 456310001 248 00:1f.0 202 4564bf001 Cc: Lu Baolu <baolu.lu@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: Ashok Raj <ashok.raj@intel.com> Cc: Jacob Pan <jacob.jun.pan@linux.intel.com> Co-Developed-by: Gayatri Kammela <gayatri.kammela@intel.com> Signed-off-by: Gayatri Kammela <gayatri.kammela@intel.com> Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Sohil Mehta <sohil.mehta@intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-09-25iommu/vt-d: Add debugfs support for Interrupt remappingSohil Mehta1-0/+108
Debugfs extension for Intel IOMMU to dump Interrupt remapping table entries for Interrupt remapping and Interrupt posting. The file /sys/kernel/debug/iommu/intel/ir_translation_struct provides detailed information, such as Index, Source Id, Destination Id, Vector and the IRTE values for entries with the present bit set, in the format shown. Remapped Interrupt supported on IOMMU: dmar1 IR table address:85e500000 Entry SrcID DstID Vct IRTE_high IRTE_low 24 01:00.0 00000001 24 0000000000040100 000000010024000d 25 01:00.0 00000004 22 0000000000040100 000000040022000d Posted Interrupt supported on IOMMU: dmar5 IR table address:85ec00000 Entry SrcID PDA_high PDA_low Vct IRTE_high IRTE_low 4 43:00.0 0000000f ff765980 41 0000000f00044300 ff76598000418001 5 43:00.0 0000000f ff765980 51 0000000f00044300 ff76598000518001 Cc: Lu Baolu <baolu.lu@linux.intel.com> Cc: Jacob Pan <jacob.jun.pan@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: Ashok Raj <ashok.raj@intel.com> Co-Developed-by: Gayatri Kammela <gayatri.kammela@intel.com> Signed-off-by: Gayatri Kammela <gayatri.kammela@intel.com> Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Signed-off-by: Sohil Mehta <sohil.mehta@intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-09-25iommu/vt-d: Add debugfs support to show register contentsGayatri Kammela1-1/+122
Add Intel IOMMU debugfs extension to dump all the register contents for each IOMMU device. Example: root@OTC-KBLH-01:~# cat /sys/kernel/debug/iommu/intel/iommu_regset IOMMU: dmar0 Register Base Address: fed90000 Name Offset Contents VER 0x00 0x0000000000000010 CAP 0x08 0x01c0000c40660462 ECAP 0x10 0x0000000000f0101a GCMD 0x18 0x0000000000000000 GSTS 0x1c 0x00000000c7000000 RTADDR 0x20 0x00000004071d3800 CCMD 0x28 0x0800000000000000 FSTS 0x34 0x0000000000000000 FECTL 0x38 0x0000000000000000 FEDATA 0x3c 0xfee0100400004021 Cc: Lu Baolu <baolu.lu@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: Jacob Pan <jacob.jun.pan@linux.intel.com> Cc: Ashok Raj <ashok.raj@intel.com> Signed-off-by: Gayatri Kammela <gayatri.kammela@intel.com> Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Co-Developed-by: Sohil Mehta <sohil.mehta@intel.com> Signed-off-by: Sohil Mehta <sohil.mehta@intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-09-25iommu/vt-d: Enable base Intel IOMMU debugfs supportSohil Mehta4-0/+35
Add a new config option CONFIG_INTEL_IOMMU_DEBUGFS and do the base enabling for Intel IOMMU debugfs. Cc: Lu Baolu <baolu.lu@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: Ashok Raj <ashok.raj@intel.com> Cc: Jacob Pan <jacob.jun.pan@linux.intel.com> Co-Developed-by: Gayatri Kammela <gayatri.kammela@intel.com> Signed-off-by: Gayatri Kammela <gayatri.kammela@intel.com> Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Sohil Mehta <sohil.mehta@intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-09-25iommu/vt-d: Relocate struct/function declarations to its header filesSohil Mehta2-29/+4
To reuse the static functions and the struct declarations, move them to corresponding header files and export the needed functions. Cc: Lu Baolu <baolu.lu@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: Jacob Pan <jacob.jun.pan@linux.intel.com> Cc: Ashok Raj <ashok.raj@intel.com> Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Signed-off-by: Gayatri Kammela <gayatri.kammela@intel.com> Signed-off-by: Sohil Mehta <sohil.mehta@intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-09-25iommu/vt-d: Handle memory shortage on pasid table allocationLu Baolu2-4/+4
Pasid table memory allocation could return failure due to memory shortage. Limit the pasid table size to 1MiB because current 8MiB contiguous physical memory allocation can be hard to come by. W/o a PASID table, the device could continue to work with only shared virtual memory impacted. So, let's go ahead with context mapping even the memory allocation for pasid table failed. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=107783 Fixes: cc580e41260d ("iommu/vt-d: Per PCI device pasid table interfaces") Cc: Ashok Raj <ashok.raj@intel.com> Cc: Jacob Pan <jacob.jun.pan@linux.intel.com> Cc: Mika Westerberg <mika.westerberg@linux.intel.com> Reported-and-tested-by: Pelton Kyle D <kyle.d.pelton@intel.com> Tested-by: Mika Westerberg <mika.westerberg@linux.intel.com> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-09-25iommu/ipmmu-vmsa: Convert to SPDX identifiersKuninori Morimoto1-4/+1
This patch updates license to use SPDX-License-Identifier instead of verbose license text. Signed-off-by: Kuninori Morimoto <kuninori.morimoto.gx@renesas.com> Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be> Reviewed-by: Simon Horman <horms+renesas@verge.net.au> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-09-25iommu/fsl: Fix spelling mistake pci_endpt_partioning -> pci_endpt_partitioningColin Ian King1-3/+3
Trivial fix to spelling mistake in variable name Signed-off-by: Colin Ian King <colin.king@canonical.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-09-25iommu/rockchip: Free irqs in shutdown handlerHeiko Stuebner1-0/+6
In the iommu's shutdown handler we disable runtime-pm which could result in the irq-handler running unclocked and since commit 3fc7c5c0cff3 ("iommu/rockchip: Handle errors returned from PM framework") we warn about that fact. This can cause warnings on shutdown on some Rockchip machines, so free the irqs in the shutdown handler before we disable runtime-pm. Reported-by: Enric Balletbo i Serra <enric.balletbo@collabora.com> Fixes: 3fc7c5c0cff3 ("iommu/rockchip: Handle errors returned from PM framework") Signed-off-by: Heiko Stuebner <heiko@sntech.de> Tested-by: Enric Balletbo i Serra <enric.balletbo@collabora.com> Acked-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-09-25iommu/dma: Use fast DMA domain lookupRobin Murphy1-11/+12
Most parts of iommu-dma already assume they are operating on a default domain set up by iommu_dma_init_domain(), and can be converted straight over to avoid the refcounting bottleneck. MSI page mappings may be in an unmanaged domain with an explicit MSI-only cookie, so retain the non-specific lookup, but that's OK since they're far from a contended fast path either way. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Tested-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-09-25iommu: Add fast hook for getting DMA domainsRobin Murphy1-0/+9
While iommu_get_domain_for_dev() is the robust way for arbitrary IOMMU API callers to retrieve the domain pointer, for DMA ops domains it doesn't scale well for large systems and multi-queue devices, since the momentary refcount adjustment will lead to exclusive cacheline contention when multiple CPUs are operating in parallel on different mappings for the same device. In the case of DMA ops domains, however, this refcounting is actually unnecessary, since they already imply that the group exists and is managed by platform code and IOMMU internals (by virtue of iommu_group_get_for_dev()) such that a reference will already be held for the lifetime of the device. Thus we can avoid the bottleneck by providing a fast lookup specifically for the DMA code to retrieve the default domain it already knows it has set up - a simple read-only dereference plays much nicer with cache-coherency protocols. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Tested-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2018-09-25iommu/iova: Optimise attempts to allocate iova from 32bit address rangeGanapatrao Kulkarni1-7/+15
As an optimisation for PCI devices, there is always first attempt been made to allocate iova from SAC address range. This will lead to unnecessary attempts, when there are no free ranges available. Adding fix to track recently failed iova address size and allow further attempts, only if requested size is lesser than a failed size. The size is updated when any replenish happens. Reviewed-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Ganapatrao Kulkarni <ganapatrao.kulkarni@cavium.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
Copyright © 1996 – 2023 Jason A. Donenfeld. All Rights Reverse Engineered.