| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Pull dma-mapping updates from Marek Szyprowski:
- swiotlb:
- new configuration option for the default pool size
(Jagadeesh Pagadala)
- reduce overhead for high watermark tracking (chenhuguanshen)
- minor code cleanups and improvements (Vova Sharaienko, Honglei Huang
and Marek Szyprowski)
- add proper tracking of the shared DMA state through direct, pool and
swiotlb paths (Aneesh Kumar K.V)
This is important for confidential-computing
* tag 'dma-mapping-7.3-2026-08-24' of git://git.kernel.org/pub/scm/linux/kernel/git/mszyprowski/linux:
dma/swiotlb: decouple high watermark tracking from CONFIG_DEBUG_FS
MAINTAINERS: update tree for DMA MAPPING HELPERS
dma/swiotlb: introduce Kconfig option for compile-time default pool size
dma-direct: Improve readability of the dma_direct_map_sg() for P2PDMA case
iommu/dma: simplify dma_iova_destroy() and drop the free_iova helper
dma-coherent: use KiB in DMA allocation logs
dma-coherent: fix spacing coding style issue
* tag 'dma-mapping-7.3-2026-08-24-2' of git://git.kernel.org/pub/scm/linux/kernel/git/mszyprowski/linux: (23 commits)
swiotlb: remove unused SWIOTLB_FORCE flag
dma: swiotlb: handle set_memory_decrypted() failures
dma: swiotlb: free dynamic pools from process context
dma-direct: rename ret to cpu_addr in alloc helpers
dma-direct: select DMA address encoding from __DMA_ATTR_ALLOC_CC_SHARED
dma-direct: set decrypted flag for remapped DMA allocations
dma-direct: make dma_direct_map_phys() honor DMA_ATTR_CC_SHARED
dma-direct: Move dma_direct_map_phys() to dma/direct.c
dma-direct: pass attrs to dma_capable() for DMA_ATTR_CC_SHARED checks
dma-mapping: make dma_pgprot() honor __DMA_ATTR_ALLOC_CC_SHARED
dma: swiotlb: track pool encryption state and honor DMA_ATTR_CC_SHARED
dma: swiotlb: pass mapping attributes by reference
dma-pool: track decrypted atomic pools and select them via attrs
dma-direct: use __DMA_ATTR_ALLOC_CC_SHARED in alloc/free paths
dma-mapping: Add internal shared allocation attribute
coco: arm64: s390: powerpc: Mark secure guests with CC_ATTR_GUEST_MEM_ENCRYPT
dma-direct: swiotlb: handle swiotlb alloc/free outside __dma_direct_alloc_pages
s390: Expose protected virtualization through cc_platform_has()
swiotlb: Preserve allocation virtual address for dynamic pools
dma: free atomic pool pages by physical address
...
|
|
Pull iommufd updates from Jason Gunthorpe:
"One small feature this cycle, the noiommu mode is useful in
single-purpose VMs running something like DPDK. It avoids the double
translation overhead and it seems to be commonly used with some hacks.
Summary:
- Formal API for "no iommu" mode in VFIO. iommufd now works in this
environment and provides page pinning and phyiscal address services
to userspace. This avoids nasty fragile tricks with mprotect and
pgmap
- Fix sykzaller crash racing change_process with map_pages
- Various skyzkaller/AI fixes for the selftests"
* tag 'for-linus-iommufd' of git://git.kernel.org/pub/scm/linux/kernel/git/jgg/iommufd:
iommufd: Fix UAF in selftest IOPF reporting
iommu/iommufd: Fix NULL pointer deref in iommufd_ioas_change_process when racing with iopt_map_file_pages
Documentation: Update VFIO NOIOMMU mode
vfio: Enable cdev noiommu mode under iommufd
iommufd: Add an ioctl to query PA from IOVA for noiommu mode
iommufd: Allow binding to a noiommu device
iommufd: Move igroup allocation to a function
iommufd: Support a HWPT without an iommu driver for noiommu
iommufd: Simplify iommufd_device_remove_vdev()
iommufd: Fix grammar and spelling in comments
iommu: Fix dev_iommu memory leak when device_add fails in iommu_mock_device_add
iommufd/selftest: Fix dmabuf leak in iommufd_test_dmabuf_get()
iommufd/selftest: Avoid selftest dirty bitmap size wrap
|
|
Pull iommu updates from Joerg Roedel:
"ARM SMMUv2:
- Device-tree binding updates for Qualcomm Eliza, Maili, Shikra and
IPQ9650 SoCs
- Add support for Qualcomm SM8450
- Numerous fixes for lifetime and ordering issues found by Sashiko in
the Qualcomm driver
ARM SMMUv3:
- Fix interrupt type in device-tree binding example for NVIDIA CMDQV
- Numerous fixes for issues identified by Sashiko in the NVIDIA CMDQV
driver
- Work around TLB erratum T264-SMMU-3 on Tegra264 by repeating the
invalidation sequence
- Add support for HAFT (hardware access flag in table entries) when
using SVA
- Probe for 52-bit addressing with a page size smaller than 64k
('DS') but don't do anything with it for now
- Minor driver improvements (remove sort_nonatomic(), use
readl_relaxed_poll_timeout_atomic(), fix IOPF teardown ordering)
Intel VT-d:
- Consolidation of complex enablement logic into a clean,
priority-based state machine
- Support for the DMA_REMAP_OPT_OUT flag from the VT-d v5.2
specification
- An update to cache_tag_flush_devtlb_psi() to use full-range
constants instead of modifying shared variables for
CACHE_TAG_NESTING_DEVTLB
- A fix for the UCTP context-table slot when copying root entries
- Fixes for several pre-existing issues reported by Sashiko
- General code cleanup and refinement
AMD IOMMU:
- Add SNP page-mode-0 support, enabling passthrough, v2 DMA page
tables and host SVA on supporting systems
- Fix invalid PPR handling, COMPLETE_PPR responses and guest-mode
reporting
- Improve Southbridge IOAPIC validation and remove the dependency on
hard-coded device IDs
- Fix PCI-device lifetime, debugfs and diagnostic issues
IOMMU core and IOMMUFD:
- Restore serialization of the shared MSI-page list
- Fix SVA-handle publication and several IOMMUFD reference and error
path leaks
- Return the expected zero result for invalid generic page-table
translations
- Allocate per-CPU IOVA magazines lazily to reduce memory use on
large systems
PCI ATS:
- Make VF support checks account for the associated PF and validate
that VF and PF Smallest Translation Unit settings agree
Platform drivers:
- Fix Qualcomm runtime-PM, probe unwind, fault reporting and page
table initialization races
- Rework Rockchip state handling and fix clock, probe and stale-fault
handling
- Fix smaller issues in the MSM and MediaTek drivers
Device-tree bindings:
- Add new Qualcomm SMMU compatibles, convert the OMAP IOMMU binding
to YAML, and fix the Tegra264 CMDQV interrupt example
Various smaller cleanups, documentation fixes and a Rust IOMMU
safety/readability improvement"
* tag 'iommu-updates-v7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/iommu/linux: (93 commits)
iommu/amd: Add SNP page mode 0 support
iommu/amd: Fix GN bit setting in COMPLETE_PPR_REQUEST command
iommu/amd: Rate limit INVALID_PPR_REQUEST error logging
iommu/amd: Fix missing CMD_COMPLETE_PPR response for invalid PPR requests
iommu/amd: Introduce PPR_TAG_LAST_PAGE() macro
iommu/amd: Fix incorrect device ID in invalid PASID error message
iommu/vt-d: Flush context cache with correct SID when tearing down aliases
iommu/vt-d: Tear down scalable-mode context on probe failure
iommu/vt-d: Fix iopf_refcount leak on RID domain replacement
iommu/vt-d: Clear Present bit before tearing down copied context entry
iommu/vt-d: Fix copied_tables bitmap leak on error in copy_translation_tables
iommu/vt-d: Cache max domain ID to avoid redundant calculation
iommu/vt-d: Support the new DMA_REMAP_OPT_OUT flag bit
iommu/vt-d: Remove dmar_disabled
iommu/vt-d: Remove the 'force_on' variable
iommu/vt-d: Call dmar_can_force_on() for tboot opt-in
iommu/vt-d: Use dmar_can_force_on() for platform opt-in
iommu/vt-d: Consolidate dmar policy management and force_on logic
iommu/vt-d: Remove dead code when CONFIG_INTEL_IOMMU is not set
iommu/vt-d: Force requesting ACS when tboot is enabled
...
|
|
Pull driver core updates from Danilo Krummrich:
"container_of:
- Apply typeof_member(), remove the local __mptr variable to
eliminate variable shadowing warnings on nested container_of()
calls, and remove unnecessary parentheses
core:
- Add driver name to probe debug print for initcall_debug
- Avoid repeatedly printing the same 'Fixed dependency cycle' log
- Unwind device_add() on attribute creation failure in
attribute_container_add_class_device()
- Remove statistics group if encryption group creation fails in
transport_add_class_device()
debugfs:
- Fix lockdown check for mmap_prepare()
- Warn if file creation failed due to uninitialized debugfs
device property:
- Implement fw_devlink support for software nodes by adding
software_node_add_links(), which creates fwnode links from
DEV_PROP_REF properties to enable automatic probe ordering. Add
kunit-managed fwnode helpers and test coverage
- Fix infinite loop in fwnode_for_each_child_node() when the
secondary fwnode has more than one child. Add test cases
- Fix out-of-bounds access in software_node_get_reference_args() when
called with index -1 (UINT_MAX)
- Refactor to use RAII approach with __free()
- Add Bartosz Golaszewski as software node reviewer
firmware loader:
- Fix race where a sysfs fallback request can complete before being
queued as pending, leading to a use-after-free on the next fallback
request
- Reject 0-size built-in firmware and fail the build on empty
firmware files in CONFIG_EXTRA_FIRMWARE
kobject:
- Provide __KOBJ_ATTR() and __KOBJ_ATTR_RO/WO() initialization macros
and allow the constification of kobject attributes, enabling them
to reside in read-only memory
platform:
- Provide platform_device_set_of_node(), platform_device_set_fwnode(),
and platform_device_set_of_node_from_dev() helpers that encapsulate
firmware node reference counting for dynamically allocated platform
devices
Convert all in-tree users that manually assigned dev.of_node or
dev.fwnode, fixing a pre-existing refcount bug in powermac. Switch
to counting references of all firmware node types, not only OF
nodes
- Unify the release path for dynamically allocated platform devices
by removing platform_device_release_full(). Amend the fwnode setter
API contract to warn if a primary software node is overwritten. Add
KUnit tests for correct software node removal on device
unregistration
Rust:
- Auxiliary:
- Add registration_data_with() closure-based API for invariant
ForLt types
- Debugfs:
- Migrate BinaryWriter and BinaryReaderMut trait requirements
from kernel::transmute traits to zerocopy traits
- Device:
- Add BoundInternal device context and InternalBoundContext trait
for bus abstractions that need internal access to a bound
device.
- Make the lifetime on Core and CoreInternal invariant to prevent
coercion to shorter lifetimes
- Devres:
- Fix race between concurrent revokers where the losing revoker
could return before the winning revoker finished dropping the
inner data, causing use-after-free.
- Ensure revocation is complete before the device finishes
unbinding by making the synchronization bidirectional.
- Add DevresLt<F: ForLt>, a wrapper around Devres that shortens
'static back to the caller's borrow scope. Implement ForLt and
CovariantForLt for Bar, IoMem, and ExclusiveIoMem
- Driver:
- Switch from index-based to pointer-based device ID info lookup,
storing static references in driver_data. Centralize device ID
handling in device_id.rs, removing the open-coded ACPI/OF
matching logic and duplicate ID table from driver.rs
- I/O:
- Make I/O regions typed (with a dynamically-sized Region type
for the existing untyped case), create view types representing
subregions of a mapped I/O region, and add io_project!() for
safely creating subviews.
- Split Io into a base trait (IoBase) and an extension trait (Io)
with a blanket implementation, preventing implementers from
overriding provided methods that unsafe code relies on.
- Add a SysMem backend for shared system memory with volatile
access, and make Coherent implement Io via an I/O view type.
Add IoSysMap as sum type of Mmio and SysMem. Add copying
methods (memcpy_{from,to}io()) and read_val()/write_val() for
typed access.
- Replace dma_read!()/dma_write!() with io_read!()/io_write!()
for primitives and copying methods for aggregates; drop the old
macros. Convert nova-core to use I/O projection.
- Fix internal shortcut rule dispatch in the register!() macro,
remove unused rule arguments, and use path fragments for alias
destinations
- IRQ:
- Make irq::Registration compatible with lifetime-bound drivers
by removing the 'static bound on Handler/ThreadedHandler and
replacing Devres<RegistrationInner> with direct
request_irq()/free_irq() calls. Handlers can now directly own
lifetime-bound device resources
- PCI:
- Convert IrqVectorRegistration to a lifetime-annotated owning
type, giving drivers explicit control over the allocation
lifetime. IrqVector embeds a resolved IrqRequest, making the
conversion infallible. Remove the redundant
request_irq()/request_threaded_irq() wrappers from pci::Device.
- Add pci_irq_type() C helper and expose it via irq_type() on
IrqVectorRegistration and IrqVector, returning PCI_IRQ_MSIX,
PCI_IRQ_MSI, or PCI_IRQ_INTX.
- Mark pci::Device refcount methods inline
- Serdev:
- Add Rust abstractions for the serial device bus, including
serdev::Driver trait, serdev::Device wrapping struct
serdev_device, and serdev::Adapter implementing
RegistrationOps. Includes a sample driver. Markus Probst takes
over as serdev maintainer for both C and Rust code
- Misc:
- Split ForLt into a base trait (providing the Of<'a> GAT) and an
unsafe CovariantForLt subtrait guaranteeing covariance,
enabling invariant types (e.g. those containing Mutex<&'bound T>)
to participate in the ForLt abstraction.
- Fix Coherent read past EOF returning -ERANGE instead of zero.
- Fix firmware example UB by avoiding null-pointer ARef
misc:
- Avoid iattr allocation in kernfs listxattr by using
kernfs_iattrs_noalloc().
- Unregister SoC bus on early device registration failure.
- Remove unused DMA_FENCE_TRACE Kconfig symbol.
- Fix /sys/module path in comment.
- Refactor ISA bus init to remove nested blocks.
- Remove redundant nodemask clears in numa_init().
- Add kernel-doc for fwnode_operations and sys_soc.h, mark
internal property data as private for kernel-doc, and add
property.h/fwnode.h to driver-api infrastructure docs.
- Add MAINTAINERS entry for sys_soc.h"
* tag 'driver-core-7.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/driver-core/driver-core: (129 commits)
rust: pci: expose the allocated interrupt type
PCI: Add pci_irq_type() to query the allocated interrupt type
rust: pci: remove request_irq() and request_threaded_irq() from Device
rust: pci: resolve IRQ in index() and embed IrqRequest in IrqVector
rust: pci: convert IrqVectorRegistration to a lifetime-managed owning type
kernfs: avoid iattr allocation in listxattr
rust: serdev: use ThisModule::as_ptr() instead of field access
ACPI/IORT: use platform_device_set_fwnode()
ACPI/APMT: use platform_device_set_fwnode()
firmware_loader: do not queue completed sysfs fallback requests
rust: pci: Mark Device refcount methods inline
rust: irq: make Registration compatible with lifetime-bound drivers
rust: net/phy: remove expansion from doc
rust: dma: return zero for Coherent reads past EOF
rust: io: register: use path fragment for alias destination
rust: io: register: remove unused rule arguments
rust: io: register: dispatch shortcut rules internally
MAINTAINERS: add sys_soc.h to DRIVER CORE
rust: debugfs: remove unsafe blocks from traits impl for Vec
rust: debugfs: migrate debugfs traits requirements to zerocopy
...
|
|
Pull arm64 updates from Will Deacon:
"There's a reasonable amount of stuff here, including a bunch of
updates to the perf PMU drivers and some MPAM updates to expose the
memory bandwidth counters via resctrl.
On the architecture side, some highlights include support for BBML3
and steps towards support for an architectural NMI solution, all
wrapped up in a web of fixes for latent issues identified by Sashiko.
ACPI:
- Combine reads of AMU counters into a single FFH feedback counter op
Confidential computing:
- Fix smp_processor_id() in preemptible context when retrieving an
attestation token inside a realm
- Convert pKVM over to a "CC platform"
- Clean-up our SWIOTLB configuration in preparation for reworking the
handling of encrypted/decryped DMA buffers in the dma-mapping tree
CPU errata handling:
- Work around broken device memory ordering on NVIDIA Olympus cores
- Fix broken 'nospectre_bhb' command-line option
- Select the idle loop backend instruction on the command-line
CPU features:
- Replace our BBML2-noabort feature with the new architectural BBML3
feature
- Disable in-kernel BTI for recent versions of Clang due to issues
with livepatch that are still being investigated
- Clean-up documentation describing which ID register fields are
exposed to userspace
Interrupts:
- Preliminary work towards supporting FEAT_NMI, which cleans up our
IRQ entry code and fixes some latent issues with pseudo-NMI
- Support for an SDEI backend to trigger an NMI backtrace
Memory management:
- Treat all devices as coherent when CLIDR_EL1.LoC == 0
- Fix no-map handling of sub-page-sized regions
- Second attempt at unmapping the linear aliases of the kernel data
and bss sections
- Fix EFI runtime calls when software-PAN is enabled
Miscellaneous:
- Add Mark Rutland as a reviewer!
- Tidy-up our futex cmpxchg logic when using the new LSUI
instructions
- Drop the requirement on DYNAMIC_FTRACE_WITH_CALL_OPS when
selecting HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS
- Fix a false-positive KCSCAN splat in the delay loop
- Use a portable typedef for 128-bit scalar types in our UAPI headers
- Non-critical fixes for Sashiko reports all over
MPAM:
- Hook MPAM memory bandwidth counters into resctrl's counter
assignment interface
- Fix a quirk in the MPAM bandwidth counting on Nvidia T241 so that
it also applies to 63 bit counters
Perf:
- Workarounds for hardware issues in the CMN-S3 PMU (Graviton 5) and
CPU PMU (NVIDIA Olympus again!)
- Add support for the DDR PMU on Marvell CN20K SoCs
- Add support for Picoheart implementations of the DCW PCIe PMU
- Add support for Channel/Rank/Bank filtering in the CXL PMU driver
- Add support for 64-bit counters in the CSPMU device
- Add support for revision 2 of the CMN S3 PMU
Ptrace:
- Fix a decade-old bug in our handling of seccomp and tracing on
syscall entry
- Fix regset handling for inactive SVE and SSVE registers
Selftests
- Add some tests for the decade-old bug that we just tried to fix in
our syscall entry path
- Fix SVE test crash on SME-only CPUs"
* tag 'arm64-upstream' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux: (95 commits)
arm64/efi: Avoid voluntary preemption with efi_mm installed
arm64: bti: Disable in-kernel BTI with recent versions of Clang
arm64: entry: Avoid unnecessary local_irq_disable() on kernel exit
irqchip/gic-v3: make the unmasking of pseudo-NMIs explicit when handling IRQs
arm64: Disable KCSAN instrumentation in delay.o
arm_mpam: Disable driver unbind to avoid UAF
arm_mpam: Fix a NULL pointer dereference on unbinding after an error interrupt
perf: arm_pmuv3: Zero initialize hw_id branch stack field
arm64: mm: Unmap kernel data/bss entirely from the linear map
iommu/arm-smmu-v3-sva: Use system_supports_bbml3() to detect CPU feature
perf/arm-cmn: Support CMN S3 r2
perf/arm-cmn: Plumb in new filter types
perf/arm-cmn: Refactor event filter data
perf/arm-cmn: Refactor event filter programming
perf/arm-cmn: Rename filter variables for clarity
arm64: mm: fix accidental linear mapping of no-map reserved memory
tools: Ensure tools copy of linux/filter.h exports the UAPI
kselftest/arm64: Fix abi test compilation errors
arch: arm64: add early_param idle=<wfi|yield|nop>
arm64: entry: mask DAIF before returning from C EL1 handlers
...
|
|
IOMMUFD selftest TRIGGER_IOPF borrows an attach handle from
group->pasid_array without synchronizing against PASID detach,
then a concurrent iommu_report_device_fault() can dereference
that borrowed handle's domain pointer after the detach erases
the handle and frees the backing struct iommufd_attach_handle.
TRIGGER_IOPF then dereferences the freed handle, causing a UAF.
Fix by adding a iopf_rwsem in mock_dev to follow the expected design
of a real driver. Hold its read side across the whole
iommu_report_device_fault() call, and its write side around every
path that attaches, detaches, or replaces a device domain.
This can block new reports and drains in-flight reports before an old
attach handle or the IOPF fault parameter can be removed.
Also take the write side while registering a mock device, since
it can invoke the mock driver's default-domain attach callback.
Closes: https://lore.kernel.org/all/D5E3AA41600B2056+f4e15662-bd2b-43ea-91cb-518de429e72c@smail.nju.edu.cn/
Fixes: ddee19971081 ("iommufd/selftest: Add IOPF support for mock device")
Cc: stable@vger.kernel.org
Suggested-by: Jason Gunthorpe <jgg@ziepe.ca>
Assisted-by: Codex:gpt-5.6-terra
Signed-off-by: Peiyang He <peiyang_he@smail.nju.edu.cn>
Link: https://patch.msgid.link/38C8DF0A118B7176+20260811095551.2756745-1-peiyang_he@smail.nju.edu.cn
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
iommufd_ioas_change_process() iterates every IOAS area while only
holding every IOAS iova_rwsem, so it assumes every area has a non-NULL
pages pointer. That assumption can be false when it runs concurrently
with iopt_map_file_pages().
iopt_map_pages() executes in two phases. It first creates the area and
inserts it into the interval tree under iova_rwsem, with area->pages
still NULL. It then drops iova_rwsem and later fills area->pages
under domains_rwsem. This leaves a window between area creation and
area->pages fill where a concurrent iommufd_ioas_change_process()
can observe the area and dereference a NULL area->pages pointer,
leading to a NULL pointer dereference:
BUG: kernel NULL pointer dereference, address: 00000000000000c0
#PF: supervisor read access in kernel mode
#PF: error_code(0x0000) - not-present page
PGD 4b655067 P4D 4b655067 PUD 0
Oops: Oops: 0000 [#1] SMP NOPTI
CPU: 0 UID: 0 PID: 11841 Comm: syz.1.628 Not tainted 7.1.0 #3 PREEMPT(full)
Hardware name: QEMU Ubuntu 24.04 PC v2 (i440FX + PIIX, arch_caps fix, 1996), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
RIP: 0010:iommufd_ioas_change_process+0x419/0xd50 drivers/iommu/iommufd/ioas.c:538
Code: 48 89 c3 48 85 c0 0f 84 cc 00 00 00 e8 10 f5 cb fd 48 8d 7b 68 e8 a7 b5 eb fd 48 8b 6b 68 48 8d bd c0 00 00 00 e8 17 b2 eb fd <8b> ad c0 00 00 00 bf 01 00 00 00 89 ee e8 85 ef cb fd 83 fd 01 74
RSP: 0018:ffffc90015c17d28 EFLAGS: 00010246
RAX: ffff8880186d5328 RBX: ffff88801d25e240 RCX: 0000000080000000
RDX: 00000000000002d7 RSI: ffffffff83ba9e10 RDI: 00000000000000c0
RBP: 0000000000000000 R08: ffffffff8e781eb8 R09: 0000000000000000
R10: 00000000000000c0 R11: ffffffff83ba9e29 R12: ffff88802e216008
R13: ffff88802e216000 R14: 0000000000000001 R15: 0000000000000000
FS: 00007f4aea3f66c0(0000) GS:ffff8880b1fa1000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000000000c0 CR3: 000000004b75c000 CR4: 0000000000350ef0
Call Trace:
<TASK>
iommufd_fops_ioctl+0x287/0x400 drivers/iommu/iommufd/main.c:533
vfs_ioctl fs/ioctl.c:51 [inline]
__do_sys_ioctl fs/ioctl.c:597 [inline]
__se_sys_ioctl fs/ioctl.c:583 [inline]
__x64_sys_ioctl+0x120/0x170 fs/ioctl.c:583
x64_sys_call+0x1092/0x1fb0 arch/x86/include/generated/asm/syscalls_64.h:17
do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
do_syscall_64+0x10a/0x680 arch/x86/entry/syscall_64.c:94
entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f4aec1a82bd
Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f4aea3f6018 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 00007f4aec436090 RCX: 00007f4aec1a82bd
RDX: 0000200000000180 RSI: 0000000000003b92 RDI: 0000000000000003
RBP: 00007f4aec250295 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007f4aec436128 R14: 00007f4aec436090 R15: 00007ffd04ef23e0
</TASK>
Modules linked in:
CR2: 00000000000000c0
---[ end trace 0000000000000000 ]---
RIP: 0010:iommufd_ioas_change_process+0x419/0xd50 drivers/iommu/iommufd/ioas.c:538
Code: 48 89 c3 48 85 c0 0f 84 cc 00 00 00 e8 10 f5 cb fd 48 8d 7b 68 e8 a7 b5 eb fd 48 8b 6b 68 48 8d bd c0 00 00 00 e8 17 b2 eb fd <8b> ad c0 00 00 00 bf 01 00 00 00 89 ee e8 85 ef cb fd 83 fd 01 74
RSP: 0018:ffffc90015c17d28 EFLAGS: 00010246
RAX: ffff8880186d5328 RBX: ffff88801d25e240 RCX: 0000000080000000
RDX: 00000000000002d7 RSI: ffffffff83ba9e10 RDI: 00000000000000c0
RBP: 0000000000000000 R08: ffffffff8e781eb8 R09: 0000000000000000
R10: 00000000000000c0 R11: ffffffff83ba9e29 R12: ffff88802e216008
R13: ffff88802e216000 R14: 0000000000000001 R15: 0000000000000000
FS: 00007f4aea3f66c0(0000) GS:ffff8880b1fa1000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000000000c0 CR3: 000000004b75c000 CR4: 0000000000350ef0
----------------
Code disassembly (best guess):
0: 48 89 c3 mov %rax,%rbx
3: 48 85 c0 test %rax,%rax
6: 0f 84 cc 00 00 00 je 0xd8
c: e8 10 f5 cb fd call 0xfdcbf521
11: 48 8d 7b 68 lea 0x68(%rbx),%rdi
15: e8 a7 b5 eb fd call 0xfdebb5c1
1a: 48 8b 6b 68 mov 0x68(%rbx),%rbp
1e: 48 8d bd c0 00 00 00 lea 0xc0(%rbp),%rdi
25: e8 17 b2 eb fd call 0xfdebb241
* 2a: 8b ad c0 00 00 00 mov 0xc0(%rbp),%ebp <-- trapping instruction
30: bf 01 00 00 00 mov $0x1,%edi
35: 89 ee mov %ebp,%esi
37: e8 85 ef cb fd call 0xfdcbefc1
3c: 83 fd 01 cmp $0x1,%ebp
3f: 74 .byte 0x74
Fix by mirroring the handling logic in iopt_unmap_iova_range(),
i.e. return -EBUSY if area->pages is NULL.
Fixes: 829ed626499c ("iommufd: Add IOMMU_IOAS_CHANGE_PROCESS")
Link: https://patch.msgid.link/r/7344B7A7B9957A65+20260806045059.1884737-1-peiyang_he@smail.nju.edu.cn
Cc: stable@vger.kernel.org
Signed-off-by: Peiyang He <peiyang_he@smail.nju.edu.cn>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
|
|
Newer AMD IOMMUs supports DTE[Mode]=0 for SNP-enabled system. This is
detected using new feature bit (EFR2[SNP_Page_Mode_0_Sup]). If this
feature is enabled, then IOMMU supports:
- Passthrough mode (i.e. iommu=pt)
- amd_iommu=pgtbl_v2: Forcing Linux DMA-API to use IOMMU v2 page table
- Setting up device for SVA mode in the host.
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Reviewed-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Tested-by: Amandeep Kaur Longia <AmandeepKaur.Longia@amd.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
The GN bit in the COMPLETE_PPR_REQUEST command indicates whether the
device is operating under a guest (v2) page table. Currently,
dev_data->pri_tlp is incorrectly used to derive this bit. However,
pri_tlp indicates whether the device uses PRI TLP which is unrelated
to page table mode.
Fix this by refactoring amd_iommu_complete_ppr() into a static internal
helper __amd_iommu_complete_ppr() that accepts an explicit 'gn' argument.
The amd_iommu_complete_ppr() wrapper then derives the GN bit correctly
from the device's active protection domain via pdom_is_v2_pgtbl_mode().
In the INVALID_PPR_REQUEST error handling path, the device's protection
domain may not be accessible, so derive the GN bit directly from the
EVENT_FLAG_PPR_GN flag in the event log entry instead.
While at it, change the 'gn' parameter type in build_complete_ppr()
from u8 to bool to better reflect its semantics.
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Reviewed-by: Ankit Soni <Ankit.Soni@amd.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
The amd_iommu_report_ppr_err() function logs an error message for every
INVALID_PPR_REQUEST event. Under certain fault conditions, a misbehaving
or malicious device can flood the IOMMU event log with PPR faults, causing
the kernel log to be overwhelmed with repeated error messages.
Switch from dev_err() to dev_err_ratelimited() to suppress duplicate
messages when INVALID_PPR_REQUEST events occur at a high rate.
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Reviewed-by: Ankit Soni <Ankit.Soni@amd.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
The AMD IOMMU spec, requires the host to respond with a CMD_COMPLETE_PPR
command when an EVENT_TYPE_INV_PPR_REQ event is received with the RX bit
cleared. This response was missing in the current implementation, leaving
invalid PPR requests unacknowledged.
Introduce amd_iommu_report_ppr_err() to handle EVENT_TYPE_INV_PPR_REQ
events. The new function logs the invalid PPR request and when the RX
bit is cleared, sends CMD_COMPLETE_PPR response.
Reported-by: Gaultier Delbarre <Gaultier.Delbarre@amd.com>
Co-developed-by: Wei Huang <wei.huang2@amd.com>
Signed-off-by: Wei Huang <wei.huang2@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Reviewed-by: Ankit Soni <Ankit.Soni@amd.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
The PPR tag field (PPRtag) encodes two distinct fields: the 9-bit tag
value (bits 8-0) and the last-page indicator L bit (bit 9).
Fix PPR_TAG() to mask only the 9-bit tag field and introduce
PPR_TAG_LAST_PAGE to explicitly extract the L bit. This way it becomes
easy to read.
Cc: Wei Huang <wei.huang2@amd.com>
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Reviewed-by: Ankit Soni <Ankit.Soni@amd.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
The IO page fault notifier handler logs pdev->dev.id when reporting an
invalid PASID, but pdev->dev.id is the kernel-internal device ID and
not the IOMMU device ID (BDF). Use dev_data->devid instead, which
reflects actual devid.
Fixes: 978d626b8f1a ("iommu/amd: Add IO page fault notifier handler")
Signed-off-by: Vasant Hegde <vasant.hegde@amd.com>
Reviewed-by: Ankit Soni <Ankit.Soni@amd.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
domain_context_clear_one() and device_pasid_table_teardown() are both
invoked once per DMA alias of a device. Each function locates the context
entry using the bus/devfn pair provided by the pci_for_each_dma_alias()
callback, then calls intel_context_flush_no_pasid(), which constructs a
device-selective context-cache invalidation from info->bus and
info->devfn (that is, always the requester ID of the device itself).
As a result, for every alias other than the device’s own RID, the context
entry that was just cleared in memory is never invalidated in the context
cache. Hardware may continue using that stale cached entry. In the
scalable-mode teardown path, intel_pasid_free_table() can then free the
PASID directory still referenced by that stale entry, allowing the IOMMU
to walk freed memory.
Fix this by passing the source ID of the entry being torn down to
intel_context_flush_no_pasid(), instead of deriving it from @info.
Fixes: f90584f4beb84 ("iommu/vt-d: Add helper to flush caches for context change")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260602233426.357499-1-baolu.lu%40linux.intel.com
Assisted-by: Claude:claude-opus-5
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Reviewed-by: Samiullah Khawaja <skhawaja@google.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
intel_pasid_setup_sm_context() walks a PCI device’s DMA aliases via
pci_for_each_dma_alias() and programs a scalable-mode context entry for
each RID. For a device with a dma_alias_mask, the callback is invoked
once for the device’s own RID and once for each alias bit, all with the
same pci_dev, so device_pasid_table_setup() runs for multiple RIDs.
pci_for_each_dma_alias() stops at the first callback error. Therefore, a
failure partway through the walk can leave context entries for already
processed RIDs present and still pointing to the device’s PASID table.
On this error path, intel_iommu_probe_device() currently jumps directly
to intel_pasid_free_table(), which frees the PASID table without
first tearing down those context entries. The IOMMU may then walk a
present context entry whose PASID table pointer references freed
memory.
intel_iommu_release_device() already performs teardown before freeing the
table. Apply the same ordering on the probe failure path.
device_pasid_table_teardown() safely handles RIDs that were never
programmed: iommu_context_addr() returns NULL when no context table has
been allocated, and clearing the Present bit of an already non-present
entry is a no-op. So unwind is safe for both the alias that failed and
any aliases not yet reached.
Fixes: 301f1a80487fd ("iommu/vt-d: Setup scalable mode context entry in probe path")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260602233426.357499-1-baolu.lu%40linux.intel.com
Assisted-by: Claude:claude-opus-5
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
intel_iommu_attach_device() enables IOPF for the new domain but never
disables it for the old one. device_block_translation(), called at the
start of the function, tears down translation but does not touch any IOPF
state; blocking_domain_attach_dev() has to call iopf_for_domain_remove()
explicitly before invoking it for exactly this reason.
identity_domain_attach_dev() has the same problem. Its comment claims
that no PRI handling is needed because the device has been put in the
blocking state, but the blocking state and the IOPF reference count are
independent of each other.
As a result, replacing a domain that has an iopf_handler with another
domain at RID level leaks a reference in info->iopf_refcount. The count
never drops back to zero, so iopf_queue_remove_device() is never called
and iommu_disable_pci_pri() triggers its WARN_ON(info->iopf_refcount)
when the device is released.
The PASID paths already handle this correctly by way of
iopf_for_domain_replace(); convert the two RID paths to do the same.
Using the replace helper rather than a bare remove keeps the enable
before the disable, so the reference count does not transiently reach
zero and evict the device from the IOPF queue.
Fixes: 17fce9d2336d ("iommu/vt-d: Put iopf enablement in domain attach path")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260602233426.357499-1-baolu.lu%40linux.intel.com
Assisted-by: Claude:claude-opus-5
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
copied_context_tear_down() zeroes the 128-bit context entry with
context_clear_entry() while the Present bit is still set, and only then
issues the context-cache and IOTLB invalidations. This leaves a window
in which hardware can fetch a torn entry, with some fields already zeroed
while Present is still set, leading to unpredictable behaviour or
spurious faults. While x86 provides strong write ordering, the compiler
may reorder the writes to the two 64-bit halves of the entry, and the
hardware fetch is not guaranteed to be atomic with respect to multiple
CPU writes.
There is no cacheline flush before the invalidation either, so on an
IOMMU without coherent access to the context table the zeroed entry may
not be visible to hardware at the point the invalidation is submitted.
Apply the same ownership handshake described in the VT-d spec, Section
6.5.3.3 ("Guidance to Software for Invalidations"): clear only the Present
bit, flush it out to the IOMMU, perform the invalidations, and only then
zero the remainder of the entry.
Fixes: c7191984e5aad ("iommu/vt-d: Factor out helpers from domain_context_mapping_one()")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260602233426.357499-1-baolu.lu%40linux.intel.com
Assisted-by: Claude:claude-opus-5
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
The iommu->copied_tables bitmap was introduced by the IOMMU live
update series to track which context entries have been copied from
the previous kernel. The allocation via bitmap_zalloc() was added
inside copy_translation_tables(), but the error paths were not
updated to free it:
1. When old_rt_phys is 0 (invalid root table address)
2. When memremap(old_rt_phys) fails
3. When kcalloc for ctxt_tbls fails (goto out_unmap, which only
unmaps old_rt without releasing the bitmap)
The bitmap is only cleaned up by free_dmar_iommu(), which is
called from the free_iommu error label in init_dmars(). However,
when copy_translation_tables() fails, init_dmars() does not jump
to free_iommu -- it logs the error, falls through, and continues
with the next IOMMU. As a result, copied_tables is leaked.
Fix this by converting the two early returns to goto a new
err_free_bitmap label, and by making out_unmap fall through to
it so that the bitmap is always freed on any error path. The
success path performs memunmap(old_rt) inline and returns 0
directly, since copied_tables must remain allocated for
subsequent use.
Signed-off-by: ZhaoJinming <zhaojinming@uniontech.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
The cap_ndoms() helper calculates the maximum available domain ID from
the value of capability register, which can be inefficient if called
repeatedly. Cache the maximum supported domain ID in max_domain_id field
during initialization to avoid redundant calls to cap_ndoms() throughout
the IOMMU driver.
No functionality change.
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Xu Yilun <yilun.xu@linux.intel.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
Some BIOS already provides config options to expose/hide VT-d units
as a whole to/from system software. A new demand is to allow exposing
VT-d units but requesting system software to disable DMA remapping
while sustaining interrupt remapping. This can be communicated now by
setting the new DMA_REMAP_OPT_OUT flag bit in the DMAR table, as
introduced in VT-d spec v5.2 (section 8.1, DMA Remapping Reporting
Structure).
Introduce a new off policy (DMAR_FW_OFF) for DMA_REMAP_OPT_OUT. As
the strongest off policy, it cannot be overridden by user opts or
any force_on types. If tboot is enabled in the meantime, kernel will
panic. It is user responsibility to configure BIOS properly.
One cleanup is left for future - the DMAR flag is parsed multiple
times, in detect_intel_iommu(), dmar_platform_optin() (which can be
called at run-time), etc. Caching it is a cleaner way.
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
It's replaced by dmar_policy_off() now, covering both "iommu=off"
and "intel_iommu=off". Also remove unnecessary checks on no_iommu,
leaving only one exception in intel_iommu_init() which skips debugfs
init for "iommu=off" but not "intel_iommu=off". Keep it to avoid
surprise for now.
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
The force_on variable is now redundant - same information captured
by "dmar_policy == DMAR_FORCE_ON". Replace all force_on checks
with dmar_policy_force_on().
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
So the policy of requesting ACS in detect_intel_iommu() is consistent
with that in tboot_force_iommu().
Though tboot is the strongest override so far, dmar_can_force_on() may
return false due to future extensions. In this case panic the kernel,
as is already done when failing to initialize DMA remapping for tboot.
No functional impact at this point.
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
So the policy of requesting ACS in detect_intel_iommu() is consistent
with that in platform_optin_force_iommu().
While at it, remove no_platform_optin which is unnecessary now.
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
Currently the dmar on/off is carried by multiple variables (no_iommu,
dmar_disabled, no_platform_optin, etc.) with error-prone force_on logic
scattered in multiple places.
Unify/centralize the policy/priority management for various force_on
scenarios.
No functional impact except one case - "intel_iommu=off" sets
no_platform_optin which is checked in platform_optin_force_iommu()
but not in detect_intel_iommu(), leading to ACS unnecessarily requested
when iommu could not be forced on later. Now with the unified logic
this becomes more consistent.
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
Those are leftovers and unreachable now: the entire intel directory
is built only when CONFIG_INTEL_IOMMU is set.
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
Currently the conditions of requesting ACS in detect_intel_iommu()
don't include tboot, leading to a possible misconfiguration with ACS
disabled (e.g. due to user opts) while iommu is later forced on by
tboot_force_iommu().
Fix it by checking tboot in detect_intel_iommu().
Fixes: 5d990b627537 ("PCI: add pci_request_acs")
Cc: stable@vger.kernel.org
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
If user explicitly requests to disable iommu (via "iommu=off" or
"intel_iommu=off"), there is no reason to force enabling it due
to platform opt-in (for external-facing devices). User should be
aware of any security implication of doing so.
"intel_iommu=off" implements this policy by setting no_platform_optin
to skip platform opt-in in platform_optin_force_iommu().
However, "iommu=off" (no_iommu=1) doesn't set no_platform_optin
hence is broken in this aspect:
- detect_intel_iommu() doesn't request ACS if no_iommu=1
- platform_optin_force_iommu() forces iommu on if external-facing
devices exist and no_platform_optin is not set
This leads to a bad configuration with ACS disabled while DMA
remapping is enabled.
Instead of setting no_platform_optin (will soon be removed) for
no_iommu=1, directly check no_iommu in platform_optin_force_iommu().
Fixes: 89a6079df791 ("iommu/vt-d: Force IOMMU on for platform opt in hint")
Cc: stable@vger.kernel.org
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
Simplify 'dmar_perf_latency_write()' by using the convenient
'kstrtoint_from_user()'.
Signed-off-by: Dmitry Antipov <dmantipov@yandex.ru>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
In cache_tag_flush_range(), the CACHE_TAG_NESTING_DEVTLB case modifies the
shared local variables 'addr' and 'mask' before falling through to
CACHE_TAG_DEVTLB. This causes all subsequent CACHE_TAG_DEVTLB entries in
the same loop iteration to incorrectly use the full-range flush parameters
(addr=0, mask=MAX_AGAW_PFN_WIDTH) instead of the precisely calculated PSI
range. This is not the intended behavior, as regular DEVTLB entries should
always perform targeted range-based invalidation.
Fix this by having CACHE_TAG_NESTING_DEVTLB directly call
cache_tag_flush_devtlb_psi() with the full-range constants and break,
instead of modifying shared variables and falling through. This ensures
CACHE_TAG_DEVTLB always uses the original calculated addr and mask for
precise range flush.
This change slightly affects trace_cache_tag_flush_range() behavior.
Previously, after addr/mask were overwritten, the tracepoint could record
a full-range flush even when the caller requested a narrower range.
The tracepoint should reflect caller intent. Although this helper may
widen the actual hardware invalidation range for implementation reasons,
that does not change what the caller requested, so logging the requested
range is the correct behavior.
If the actual invalidation range sent to hardware is needed, it is already
visible via the qi_submit trace event, which records the invalidation
descriptors emitted by the driver.
Signed-off-by: Guanghui Feng <guanghuifeng@linux.alibaba.com>
Signed-off-by: Guixin Liu <kanie@linux.alibaba.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
Replace bitwise OR (|) with logical OR (||) in the privilege mode
validation check. While both operators produce the same result for
boolean values (0 or 1), using logical OR is semantically correct and
makes the intent clearer.
No functional change, but improves code readability.
Signed-off-by: Li RongQing <lirongqing@baidu.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
When translation is already enabled at boot (e.g. kdump), the vt-d driver
copies context tables from the previous kernel's root table. In scalable
mode, buses that only populate the upper root half (UCTP, devfn >= 0x80)
should be written to ctxt_tbls[tbl_idx + 1] through copy_context_table().
However, the current copy path always uses tbl[tbl_idx + 0] in this situa-
tion. Since idx wraps to 0 at devfn 0x80 due to a zeroed LCTP, new_ce for
LCTP will be NULL and keep pos equals to 0. Thus, UCTP entries will be co-
pied into tbl[tbl_idx + 0] instead of tbl[tbl_idx + 1], and written after-
wards to root_entry[bus].lo instead of .hi in copy_translation_tables().
In short, devices on bus 0x80 with devfn >= 0x80 fail DMA with fault 0x39,
which will break drivers running in kernels with translation pre-enabled.
This fixes NO_PASID DMAR faults for UCTP-only buses such as:
DMAR: [DMA Read NO_PASID] Request device [80:14.0] fault addr 0xe81759000
[fault reason 0x39] SM: Present bit in Root Entry is clear
For instance, this fault yielded to locking issues between systemd and
xHCI, blocking a system's reboot after a vmcore was captured with kdump:
systemd-udevd[246]: usb3: Worker [255] processing SEQNUM=2193 is taking a long time
dracut-initqueue[277]: Timed out while waiting for udev queue to empty.
systemd-udevd[246]: usb3: Worker [255] processing SEQNUM=2193 killed
systemd-udevd[246]: usb3: Worker [255] terminated by signal 9 (KILL).
...
kdump[569]: saving vmcore complete
...
systemd-shutdown[1]: Rebooting.
INFO: task kworker/0:1:11 blocked for more than 122 seconds.
Not tainted 7.0.0-clean #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:kworker/0:1 state:D stack:0 pid:11 tgid:11 ppid:2 task_flags:0x4208160 flags:0x00080000
Workqueue: usb_hub_wq hub_event
Call Trace:
<TASK>
__schedule+0x299/0x5c0
schedule+0x27/0x80
schedule_timeout+0xbd/0x100
__wait_for_common+0x97/0x1b0
? __pfx_schedule_timeout+0x10/0x10
xhci_alloc_dev+0x9e/0x2b0
usb_alloc_dev+0x7a/0x3b0
hub_port_connect+0x285/0x960
hub_port_connect_change+0x94/0x290
port_event+0x4bb/0x840
hub_event+0x141/0x460
process_one_work+0x196/0x390
worker_thread+0x1af/0x320
? __pfx_worker_thread+0x10/0x10
kthread+0xe3/0x120
? __pfx_kthread+0x10/0x10
ret_from_fork+0x199/0x260
? __pfx_kthread+0x10/0x10
ret_from_fork_asm+0x1a/0x30
</TASK>
INFO: task systemd-shutdow:1 blocked for more than 122 seconds.
Not tainted 7.0.0-clean #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:systemd-shutdow state:D stack:0 pid:1 tgid:1 ppid:0 task_flags:0x400100 flags:0x00080000
Call Trace:
<TASK>
__schedule+0x299/0x5c0
schedule+0x27/0x80
schedule_preempt_disabled+0x15/0x30
__mutex_lock.constprop.0+0x547/0xac0
device_shutdown+0xac/0x1b0
kernel_restart+0x3a/0x70
__do_sys_reboot+0x147/0x240
do_syscall_64+0x11b/0x6a0
? handle_mm_fault+0x110/0x350
? do_user_addr_fault+0x206/0x680
? irqentry_exit+0x7a/0x4d0
entry_SYSCALL_64_after_hwframe+0x76/0x7e
RIP: 0033:0x7fe2958da917
RSP: 002b:00007ffc5c458618 EFLAGS: 00000206 ORIG_RAX: 00000000000000a9
RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe2958da917
RDX: 0000000001234567 RSI: 0000000028121969 RDI: 00000000fee1dead
RBP: 00007ffc5c458790 R08: 0000000000000069 R09: 00000000ffffffff
R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000000
R13: 0000000000000000 R14: 00007ffc5c4588b8 R15: 0000000000000000
</TASK>
INFO: task systemd-shutdow:1 is blocked on a mutex likely owned by task kworker/0:1:11.
Fixes: 091d42e43d21 ("iommu/vt-d: Copy translation tables from old kernel")
Signed-off-by: Desnes Nunes <desnesn@redhat.com>
Tested-by: Tao Liu <ltao@redhat.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Reviewed-by: Samiullah Khawaja <skhawaja@google.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
Unlike a group's default domain, which is always freshly allocated
and privately owned (iommu_group_alloc_default_domain()), VFIO type1's
legacy container merges any newly attached group into an existing
domain whenever their iommu_ops and cache-coherency enforcement match.
iommu_dma_get_msi_page() only asserts the caller's own group mutex is
held (iommu_group_mutex_assert()). On an IOMMU that publishes
IOMMU_RESV_SW_MSI, e.g. ARM SMMU, a VM with two such devices assigned
through the legacy container can have their guest drivers probe and
allocate MSIs in parallel; each host-side VFIO_DEVICE_SET_IRQS lands
on a different device fd and group mutex, but both devices' domains
are the same merged domain, so both can enter
iommu_dma_get_msi_page() concurrently and corrupt msi_page_list.
commit 288683c92b1a ("iommu: Make iommu_dma_prepare_msi() into a
generic operation") dropped the prior msi_prepare_lock on the
reasoning that "each iommu_domain is unique to a group," which holds
for default domains but not this VFIO type1 case. Restore the static
lock, since it's only guarding a corner case and will likely never
be contended.
iommufd avoids the equivalent problem by having its own callers
(iommufd_sw_map_msi()) take a ctx-wide sw_msi_lock before ever
reaching the shared list. VFIO type1 can't mirror that since it
dispatches to iommu_dma_sw_msi() which is outside VFIO's jurisdiction.
Fixes: 288683c92b1a ("iommu: Make iommu_dma_prepare_msi() into a generic operation")
Signed-off-by: Andrew Jones <andrew.jones@oss.qualcomm.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Nutty Liu <nutty.liu@hotmail.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
DOMAIN_NS(iova_to_phys) returns a phys_addr_t and follows the IOMMU
iova_to_phys() convention of returning 0 when there is no valid
translation.
make_range() returns a negative errno if the input IOVA cannot be
represented by this page table format. Returning that errno directly
casts it to phys_addr_t. Callers treat zero as no translation and use
non-zero values as physical addresses, so this can make them consume a
bogus physical address instead of seeing a failed translation.
Match the page-table walk failure path and return 0 for range validation
failures too.
Fixes: 9d4c274cd7d5 ("iommupt: Add iova_to_phys op")
Cc: stable@vger.kernel.org
Assisted-by: Qoder:Qwen-3.8-MAX-Preview
Signed-off-by: Shuai Xue <xueshuai@linux.alibaba.com>
Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
iommufd_take_all_iova_rwsem() takes an object reference and the
iova_rwsem write lock before storing the IOAS in the temporary ioas_list
xarray.
If xa_store() fails, the current IOAS has not been inserted into
ioas_list yet. iommufd_release_all_iova_rwsem() only unwinds IOAS
objects already present in that xarray, so it cannot release the current
IOAS.
Release the current IOAS rwsem and object reference before unwinding the
previously stored entries.
Fixes: 051ae5aa73d7 ("iommufd: Lock all IOAS objects")
Cc: stable@vger.kernel.org
Assisted-by: Qoder:Qwen-3.8-MAX-Preview
Signed-off-by: Shuai Xue <xueshuai@linux.alibaba.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
iommufd_access_notify_unmap() skips internal accesses because they do
not have an external unmap callback to invoke.
However, the current test calls iommufd_lock_obj() before checking
whether the access is internal. If iommufd_lock_obj() succeeds, the loop
then sees the internal access and continues, bypassing the matching
iommufd_put_object() used by the normal unmap path. This leaks the
object reference taken by iommufd_lock_obj().
Check for internal accesses first so skipped entries are never locked.
Fixes: 27b77ea5feaa ("iommufd/access: Bypass access->ops->unmap for internal use")
Cc: stable@vger.kernel.org
Assisted-by: Qoder:Qwen-3.8-MAX-Preview
Signed-off-by: Shuai Xue <xueshuai@linux.alibaba.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
iommu_attach_device_pasid() installs the new SVA attach handle in the
group PASID lookup before iommu_sva_bind_device() returns. A concurrent
bind can therefore find and reuse the same handle after iommu_sva_lock is
dropped.
handle->dev was initialized after dropping iommu_sva_lock. This leaves a
window where a racing bind can return a handle whose dev pointer is still
NULL. A subsequent iommu_sva_unbind_device() can then dereference it via
handle->dev->iommu_group.
Initialize handle->dev before releasing iommu_sva_lock so any visible SVA
handle is fully initialized.
Fixes: be51b1d6bbff ("iommu/sva: Refactoring iommu_sva_bind/unbind_device()")
Cc: stable@vger.kernel.org
Assisted-by: Qoder:Qwen-3.8-MAX-Preview
Signed-off-by: Shuai Xue <xueshuai@linux.alibaba.com>
Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
We need the driver-core fixes in here as well to build on top of.
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
|
|
Commit 94104e3cfa80 ("arm64: cpufeature: Rename BBML2_NOABORT as BBML3")
renamed the cpu_supports_bbml2_noabort() helper to cpu_supports_bbml3(),
as the Linux-defined "noabort" semantics have now been incorporated into
the architecture under the BBML3 feature.
Update the caller in the SMMUv3 SVA driver to use the new function.
There is a slightly oddity in that the SMMUv3 architecture already
defined BBML2 in such a way that aborts were prohibited, so we use the
BBML3 feature on the CPU to enable BBML2 in the SMMU.
Fixes: 94104e3cfa80 ("arm64: cpufeature: Rename BBML2_NOABORT as BBML3")
Signed-off-by: Will Deacon <will@kernel.org>
|
|
The number of master->num_streams per master device is typically very
small in practice. Sorting this array takes a very small amount of
time, so there is no practical risk of triggering a soft lockup that
would necessitate calling cond_resched() during the sort.
Replace sort_nonatomic() with the standard sort(). Since this is the
only remaining in-tree caller of sort_nonatomic(), this change paves
the way to eventually remove the unused sort_nonatomic() API from the
core library.
Signed-off-by: Kuan-Wei Chiu <visitorckw@gmail.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
An SMMU implementing SMMU_IDR5.DS extends the range invalidation commands:
the SCALE field grows a 6th bit, raising its maximum value from 31 to 39,
and TTL == 0b01 becomes a valid level hint for a 16KB translation granule.
Add a new ARM_SMMU_FEAT_DS feature detecting the DS bit, and widen the
CMDQ_TLBI_0_SCALE field to its architectural 6 bits. Mask the scale value
explicitly in arm_smmu_cmdq_batch_add_range(), so the range invalidation
path emits the same commands as before, keeping the pre-existing 5-bit
truncation of a scale above 31.
Also list DS as a valid IDR5 field in the iommu_hw_info_arm_smmuv3 kdoc:
iommufd has always reported the raw IDR5 register, so a VMM may conclude
from that bit alone that it can expose DS to its guest.
Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Pranjal Shrivastava <praan@google.com>
Assisted-by: Claude:claude-fable-5
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
The arm_smmu_write_reg_sync() helper is currently implemented using
readl_relaxed_poll_timeout() (that relies on usleep_range() internally)
which becomes a critical issue when used in the gerror irq handler.
If the SMMU hits a gerror and enters Service Failure Mode
(GERROR_SFM_ERR), the gerror handler calls arm_smmu_device_disable() in
hard-irq context. This becomes a problem as arm_smmu_device_disable()
inevitably calls arm_smmu_write_reg_sync() which might attempt to sleep
inside a hard-irq context.
Fix this by converting the arm_smmu_write_reg_sync to use the
readl_relaxed_poll_timeout_atomic() polling helper.
(Discovered while running Sashiko locally on another patch series).
Reported-by: Sashiko <sashiko-bot@kernel.org>
Fixes: 48ec83bcbcf5 ("iommu/arm-smmu: Add initial driver support for ARM SMMUv3 devices")
Signed-off-by: Pranjal Shrivastava <praan@google.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
Since table access flags cannot be software-managed, if process
pagetables are using HAFT then SVA must require the SMMU to support and
enable it too, otherwise page aging is liable to get out of whack.
For unbinding, we can't disable HAFT atomically with HA as might be
desired, but luckily we can get away with just not disabling HA either.
Cc: stable@vger.kernel.org
Fixes: 62df5870ebf7 ("arm64: Enable ARCH_HAS_NONLEAF_PMD_YOUNG")
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
Change swiotlb_tbl_map_single() to take the DMA mapping attributes by
reference and update the direct callers accordingly.
This is a preparatory change for a follow-up patch which updates the
attributes based on the selected swiotlb pool. Keeping the signature change
separate makes the follow-up patch easier to review.
No functional change in this patch.
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Tested-by: Michael Kelley <mhklinux@outlook.com>
Tested-by: Mostafa Saleh <smostafa@google.com>
Reviewed-by: Petr Tesarik <ptesarik@suse.com>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
Link: https://lore.kernel.org/r/20260717180442.110954-13-aneesh.kumar@kernel.org
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
|
|
Teach the atomic DMA pool code to distinguish between encrypted and
unencrypted pools, and make pool allocation select the matching pool based
on DMA attributes.
Introduce a dma_gen_pool wrapper that records whether a pool is
unencrypted, initialize that state when the atomic pools are created, and
use it when expanding and resizing the pools. Update dma_alloc_from_pool()
to take attrs and skip pools whose encrypted state does not match
__DMA_ATTR_ALLOC_CC_SHARED. Update dma_free_from_pool() accordingly.
Also pass __DMA_ATTR_ALLOC_CC_SHARED from the swiotlb atomic allocation
path so decrypted swiotlb allocations are taken from the correct atomic
pool.
Tested-by: Jiri Pirko <jiri@nvidia.com>
Tested-by: Michael Kelley <mhklinux@outlook.com>
Tested-by: Mostafa Saleh <smostafa@google.com>
Reviewed-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
Link: https://lore.kernel.org/r/20260717180442.110954-12-aneesh.kumar@kernel.org
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
|
|
The non-blocking, non-coherent allocation path uses dma_alloc_from_pool(),
which returns the allocated page and fills cpu_addr only on success.
Do not rely on cpu_addr to detect allocation failure in this path. Check
the returned page directly before using it for the IOMMU mapping.
Fixes: 9420139f516d ("dma-pool: fix coherent pool allocations for IOMMU mappings")
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Tested-by: Michael Kelley <mhklinux@outlook.com>
Tested-by: Mostafa Saleh <smostafa@google.com>
Reviewed-by: Petr Tesarik <ptesarik@suse.com>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
Link: https://lore.kernel.org/r/20260717180442.110954-4-aneesh.kumar@kernel.org
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
|
|
Pull iommufd fixes from Jason Gunthorpe:
"Several bug fixes found by tools and fuzzing:
- Incorrect domain passed during replace to ack faults
- Block the access API from using dmabuf
- Missing unlock on error unwind
- Race seeing a partially setup vdevice in the xarray
- Do not allow vdevices to have multiple stream ids in SMMUv3
- Possible UAF if racing faults with domain changes"
* tag 'for-linus-iommufd' of git://git.kernel.org/pub/scm/linux/kernel/git/jgg/iommufd:
iommu/iommufd: Fix IOPF group ownership UAF
iommu/arm-smmu-v3-iommufd: Require exactly one Stream ID for a vDEVICE
iommufd/viommu: Publish a vDEVICE only after vdevice_init() succeeds
iommufd/viommu: Release the igroup lock on the vdevice_size error path
iommufd: Reject DMABUF pages from the access pin path
iommufd: Fix wrong hwpt passed to iommufd_auto_response_faults on replace
|
|
A large amount of memory may be allocated for these magazines on
machines with a lot of IOMMU groups and CPU cores. Not all may be used
as some devices may be unused or be bound to drivers that do not use the
DMA-API. Furthermore, some drivers may not use all levels or CPUs.
Move the initialization of the loaded and prev magazines for each CPU on
the first attempt to try to insert a freed IOVA to them.
Signed-off-by: Logan Odell <loganodell@google.com>
Signed-off-by: Michal Clapinski <mclapinski@google.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
iommu_call_iopf_notifier() looks up the requester with
pci_get_domain_bus_and_slot(), which returns a PCI device with its
reference count incremented.
Neither the successful iommu_report_device_fault() path nor the abort
path drops that reference, so every handled PPR request leaks a PCI
device reference.
This is the same ownership rule that was fixed for the old iommu_v2
ppr_notifier() path by commit 6cf0981c2233 ("iommu/amd: Fix pci device
refcount leak in ppr_notifier()"), but iommu_call_iopf_notifier() was
added later as a separate PPR/IOPF notifier path.
Drop the PCI device reference after handling the PPR entry.
Fixes: 978d626b8f1a ("iommu/amd: Add IO page fault notifier handler")
Cc: stable@vger.kernel.org
Assisted-by: Qoder:Qwen-3.8-MAX-Preview
Signed-off-by: Shuai Xue <xueshuai@linux.alibaba.com>
Reviewed-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
Linus Torvalds
Peiyang He
Joerg Roedel
Vasant Hegde
Lu Baolu
ZhaoJinming
Kevin Tian
Dmitry Antipov
Guanghui Feng
Li RongQing
Desnes Nunes
Andrew Jones
Shuai Xue
Danilo Krummrich
Will Deacon
Kuan-Wei Chiu
Nicolin Chen
Pranjal Shrivastava
Robin Murphy
Aneesh Kumar K.V (Arm)
Logan Odell