laptop-kernel/virt, branch master

KVM: WARN if vCPU creation is in-progress when locking all vCPUs

2026-09-26T04:49:49Z

Now that KVM holds kvm->lock for the entirety of vCPU creation, from when created_vcpus is incremented until the new vCPU is fully onlined, WARN if the impossible happens and KVM somehow sees a discrepancy between the number of vCPUs "created" and "onlined". Signed-off-by: Sean Christopherson Tested-by: Jean-Christophe Guillain Tested-by: Naveen N Rao (AMD) Message-ID: <20260921174445.911676-8-seanjc@google.com> Signed-off-by: Paolo Bonzini

Revert "KVM: Check for duplicate vcpu_id as early as possible"

2026-09-26T04:49:49Z

Now that KVM uses kvm_get_vcpu_by_id() to check for an existing vCPU ID before doing any meaningful work, which was made possible by holding kvm->lock for the entirety of vCPU creation, revert the now-redundant "early" vCPU ID tracking. The claims about the impact of kvm->vcpu_ids on the memory footprint were a wee bit wrong: the worst case scenario isn't 256 bytes per VM, it's 256 "unsigned longs" per VM, i.e. 2048 bytes per VM. Increasing the size of "struct kvm" by 2048 nearly doubled the total size on many architectures, and tripped x86's KVM_SANITY_CHECK_VM_STRUCT_SIZE, which was added to detect this *exact* scenario, where a single change significantly increased the size of "struct kvm". I.e. attempting to build KVM with CONFIG_DEBUG_KERNEL=n fails on x86 (the build failures got missed because all build bots apparently test only CONFIG_DEBUG_KERNEL=y kernels, and maintainers' test flows were similarly lacking). This reverts commit 97d65b544f48b2ee49f6aea32145e3e7969955dc. Fixes: 97d65b544f48 ("KVM: Check for duplicate vcpu_id as early as possible") Reported-by: Jean-Christophe Guillain Closes: https://lore.kernel.org/all/56a4bc35ee605588b7cc36c8e45c12b5f3b506cb.camel@guillain.net Reported-by: Paweł S Closes: https://lore.kernel.org/all/CABD%3DWFOS4j4hDv%2BpW-eEM9HAM2q2GY_iYdAG%2BqvYcUEinUrcQQ@mail.gmail.com Tested-by: Jean-Christophe Guillain Signed-off-by: Sean Christopherson Tested-by: Naveen N Rao (AMD) Message-ID: <20260921174445.911676-7-seanjc@google.com> Signed-off-by: Paolo Bonzini

KVM: Move check for existing vCPU ID to the top of vCPU creation

2026-09-26T04:49:49Z

Now that kvm->lock is held for the entirety of vCPU creation, check for a conflicting vCPU ID at the begnning of vCPU creation, before the arch precreate() hook is invoked. This will allow reverting commit 97d65b544f48 ("KVM: Check for duplicate vcpu_id as early as possible"). For now, keep the redundant vcpu_ids tracking as a sanity check. No functional change intended (absent KVM bugs, checking vcpu_ids and walking kvm_get_vcpu_by_id() should yield the same result). Signed-off-by: Sean Christopherson Tested-by: Jean-Christophe Guillain Tested-by: Naveen N Rao (AMD) Message-ID: <20260921174445.911676-6-seanjc@google.com> Signed-off-by: Paolo Bonzini

KVM: Protect all of kvm_vm_ioctl_create_vcpu() with kvm->lock

2026-09-26T04:49:49Z

When creating a vCPU, don't drop kvm->lock to when doing the bulk of actual vCPU creation, as allowing multiple vCPUs to be created in parallel adds significant complexity in KVM (as evidenced by the many related bugs), and all known VMMs fully serialize vCPU creation. Remove all manually locking of kvm->lock from kvm_arch_vcpu_{post,}create() for obvious reasons. For many years, "everyone" has assumed that dropping kvm->lock was done for performance reasons optimization, e.g. to allow userspace to create all vCPUs concurrently for latency purposes. But as above, no known VMM does that. Looking at the history of this code, before commit 11ec28047118 ("KVM: Convert vm lock to a mutex"), kvm->lock was a spinlock. I.e. KVM *had* to drop kvm->lock when doing the bulk of vCPU creation, otherwise KVM couldn't do normal memory allocations. When kvm->lock got turned into a mutex for unrelated reasons, no one took advantage updated of the change to simplify vCPU creation. And 19 years later, everyone just assumed that KVM continued to deal with the complexity for performance reasons. Furthermore, naively parallelizing vCPU creation in userspace is likely a net negative due to the overheads of task creation. Unless a VMM carefully avoids the extra overhead related to parallelization, e.g. spawns each vCPU's thread before creating the vCPU, creating vCPUs concurrently is a net *negative* up until about ~64 vCPUs, after which the times are a wash. The absolute speed of light _is_ faster if KVM doesn't hold kvm-lock, but at vCPU counts of ~16 or less, it's probably in the noise when considering total VM creation time, as the added latency is less than 1ms up until 16 or so vCPUs. On top of all that, KVM has had a *lot* of fatal bugs (most often found by syzkaller) related to vCPUs being created while trying to do per-VM operations (basically, see every flow that locks all vCPUs). I.e. the parallel vCPU creation "support" is actively harmful as the only "use case" is for misbehaving userspace to exploit KVM bugs. Serializing vCPU creation will allow reverting commit 97d65b544f48 ("KVM: Check for duplicate vcpu_id as early as possible"), which had "minor" math error: the worst case scenario isn't "256 bytes per VM", it's "256 unsigned longs per VM", i.e. 2048 bytes per VM, which doubles the size of each VM and pushes several architectures into order-1 allocations. Signed-off-by: Sean Christopherson Tested-by: Jean-Christophe Guillain Tested-by: Naveen N Rao (AMD) Message-ID: <20260921174445.911676-5-seanjc@google.com> Signed-off-by: Paolo Bonzini

KVM: Reject attempts to lock all vCPUs if vCPU creation is in-progress

2026-09-26T04:49:49Z

Reject locking of all vCPUs if vCPU creation is in-progress, i.e. if the number of "created" vCPUs doesn't match the number of "onlined" vCPUs. It's simply not possible to guarantee that KVM has truly locked all vCPUs if one or more vCPUs are actively being created. Holding kvm->lock does prevent in-flight vCPUs from being fully onlined, but it's infeasible for common KVM to know whether or not that provides sufficient protection. In practice, this is likely a minor bug fix for the ARM and RISC-V usage of kvm_trylock_all_vcpus(), and a glorified nop for everything else. E.g. ARM's kvm_timer_vcpu_init() can race kvm_vm_ioctl_set_counter_offset() with respect to observing KVM_ARCH_FLAG_VM_COUNTER_OFFSET. Opportunistically drop x86's existing manual checks on vCPU creation being in-progress as all of x86's checks immediately precede or follow locking of all vCPUs. Leave arm64 and RISC-V alone for the moment, as their checks aren't as obviously redundant/equivalent. Signed-off-by: Sean Christopherson Tested-by: Jean-Christophe Guillain Tested-by: Naveen N Rao (AMD) Message-ID: <20260921174445.911676-2-seanjc@google.com> Signed-off-by: Paolo Bonzini

KVM: Don't pre-reserve xarray entries when storing empty/NULL attributes

2026-09-22T15:36:16Z

Skip the xarray reservation loop when clearing all memory attributes, as storing NULL only erases the entry and never needs to allocate, so no reservation (and no cleanup of a failed one) is required in that case. Suggested-by: Sean Christopherson Cc: David Ballesteros Signed-off-by: Zeng Chi Link: https://patch.msgid.link/20260921102442.1232375-1-zeng_chi911@163.com [sean: split to separate patch] Signed-off-by: Sean Christopherson

KVM: Ensure memory attributes xarray nodes are accounted to the caller's memcg

2026-09-22T15:36:06Z

Explicitly instantiate the memory attributes xarray with XA_FLAGS_ACCOUNT to ensure that all allocations are accounted to the memcg. Frustratingly, memory allocations done in the "fastpath" do not honor the passed in gfp, even for an explicit xa_reserve(). Only the rare, slow path __xas_nomem() honors the original gfp. E.g. xa_reserve(..., GFP_KERNEL_ACCOUNT) | -> ... | -> __xa_cmpxchg_raw() | -> xas_store() <== does not take @gfp | -> xas_create() | -> xas_alloc() The bug was confirmed by observing that a process in a cgroup limited to 256 MiB grew radix_tree_node slab by ~512 MiB while its memory.current stayed near 0. Fixes: 5a475554db1e ("KVM: Introduce per-page memory attributes") Cc: stable@vger.kernel.org Assisted-by: Claude-Code:claude-opus-5 Signed-off-by: David Ballesteros Link: https://patch.msgid.link/20260915175335.138547-4-davimaba.v@proton.me [sean: rewrite changelog, tag for stable] Signed-off-by: Sean Christopherson

KVM: Don't treat reserved xarray entries as having memory attributes

2026-09-22T15:31:44Z

kvm_vm_set_mem_attributes() reserves an xarray entry for every gfn in the range before storing the new attributes, so that the store loop can't fail partway through. If one of the reservations fails, e.g. with -ENOMEM, the entries that were already reserved are left in the array. That is harmless as far as xa_reserve() is concerned, as the reserved entries read back as NULL via xa_load(), but it confuses the "does this range have no attributes at all" check: if (!attrs) return !xas_find(&xas, end - 1); A reserved entry is XA_ZERO_ENTRY, not NULL, and xas_find() returns it as present. So a leftover reservation makes KVM report that a fully shared range has attributes even though kvm_get_memory_attributes() returns none for every gfn in the range. On x86, the next time mixed-attribute tracking is recomputed for the range (memslot creation, or a later attribute change that straddles the 2MiB page), hugepage_has_attrs() treats a fully shared 2MiB range as mixed and refuses to map it with a hugepage, until userspace happens to set attributes on the range again. Drop the shortcut and handle the !attrs case in the per-index loop, using xas_next_entry() to find the next non-NULL entry. xas_next_entry() is essentially an optimized xas_find(), so the effective change is that the !attrs lookup now goes through xas_retry() like the attrs != 0 case, i.e. reserved entries are skipped and retry entries restart the walk. Don't check the index when no entry is found, as the xarray leaves the xas index in a bogus state in that case; no entry simply means the rest of the range has no attributes. KVM never stores a non-NULL entry with a value of zero (clearing stores NULL), but such an entry would be returned by xas_next_entry() and trip the index check, so WARN if one is ever seen. Fixes: 5a475554db1e ("KVM: Introduce per-page memory attributes") Cc: stable@vger.kernel.org Suggested-by: Sean Christopherson Cc: David Ballesteros Signed-off-by: Zeng Chi Link: https://patch.msgid.link/20260921102442.1232375-1-zeng_chi911@163.com [sean: expand comment to elaborate on xarray APIs, split optimization out] Signed-off-by: Sean Christopherson

Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm

2026-08-25T18:48:04Z

Pull kvm updates from Paolo Bonzini: "ARM64: - Add support for 'slot' based PMU events, paired with new UAPI that compels the user to select a specific PMU implementation - Lazy save/restore of vCPU state for pKVM, along with various fixes and cleanups to the management of vCPU state between the untrusted host and pKVM hypervisor - Disable traps of EL1 registers for nested hypervisors when FEAT_NV2p1 is present, guaranteeing that EL2-specific register bits are stateful in the EL1 counterpart - Leverage FEAT_NV3 to avoid unnecessary ERET/TLBI traps when the scope of those instructions remains 'in host' (i.e. L1 kernel/userspace) - Pile of fixes for the management of the VNCR pseudo-TLB, such as under-invalidations and races with concurrent TLBIs on other vCPUs - Consolidate the non-protected and pKVM view of ICH_VTR_EL2 to a runtime-patched constant, allowing the same data to be shared with pKVM prior to dropping host privileges - Considerable pile of LLM-assisted fixes around the shop but mostly in the VGIC, our in-kernel generator of bugs (and sometimes interrupts) LoongArch: - Advertise already-supported capabilities - Some bug fixes about timer and MMIO - Some hardening about interrupt injection - Replace kvm_err() with kvm_pr_unimpl() - Add FPU/LSX/LASX test cases for selftests RISC-V: - Svadu/Zicfiss/Zicfilp FWFT support for Guest - Use try_cmpxchg for IMSIC MRIF RMW - More arch-specific tracepoints in KVM RISC-V - Eager page splitting when enabling dirty logging - Optimize hfence request handling for SMP Guests - Improve dirty log clearing by skipping zero bits in mask - Guard HFENCE range loops against overflow - CPU PM notifiers in KVM RISC-V for non-retentive idle states - Fix kernel-mode vector context save/restore for Guest s390: - Fixes for vfio-ap - Fixes for the gmap rework - Fixes for vsie - AI triggered fixes all over - diag9c tracing - code move preparation for the additional arm64 support - enable CONTEXT_ANALYSIS x86: - Perform spring cleaning on x86.{c,h} and asm/kvm_host.h, by adding regs.c (the kvm_cache_regs.h => regs.h is already applied) and msrs.{c,h}, and moving relevant code out of x86.c - Split kvm_mmu in three parts, respectively to describe the format of page tables, walking the guest page tables and building the page tables. Always use the same page table walker kvm->arch.gva_walk as the entry point to convert a guest's virtual address, where the previous code used two different kvm_mmu structs depending on whether the walk included nested EPT/NPT or not. Make page fault vmexits reuse the permission checking machinery that is used for guest page faults. This is both a cleanup and a baby step towards supporting XS/XU memory permissions - Document some of the "fun" gotchas with the APIC base when creating IRQCHIPs on x86 - Remove a defunct masterclock update from kvm_xen_shared_info_init(). It could result in incorrect kvmclock due to triggering an unnecessary switch to/from masterclock mode - Skip Xen runstate time updates if time has effectively gone backwards, so that the guest doesn't report 100% steal time for a very, very long time - Drop KVM's runtime updates of the Xen PV timing CPUID leaf, as KVM was updating the wrong sub-leaf, and upstream KVM will soon provide all the information needed by userspace to populate the CPUID field itself - Fix a bug where KVM would walk a newly created rmap without holding the rmap lock (or mmu_lock) during aging - Fix a bug where aging TDP MMU SPTEs could clobber FROZEN SPTEs - Fix a variety of #DB priority bugs - Fix a class of races related to enabling Hyper-V emulation on a vCPU after the vCPU is visible to the rest of KVM - Use static calls for nested virtualization ops - Move more KVM-internal code out of x86's kvm_host.h - Enumerate support for a variety of Zhaoxin instructions that don't require explicit virtualization - Fix missing EFER validation bugs, including in the KVM_SET_SREGS* path - Harden kvm_vcpu_map() against double-mapping and thus leaking references - Misc fixes and cleanups, e.g. for largely benign syzkaller splats x86 (Intel): - Zero a vCPU's entry in VMX's Posted Interrupt Descriptor table used for IPI virtualization when the vCPU is freed, to fix a use-after-free where hardware will write to a freed vCPU's PID - Service local TLB flushes on a failed nested VM-Enter to fix a bug where KVM could miss a TLB on a future, successful VM-Enter with the same L2 VPID - Cap the maximum value shoved into the VMX Preemption Timer to workaround an erratum that affects all existing Intel CPUs that support CPUID 0x15 - Fix VPID virtualization bugs where KVM would fail to flush hardware TLBs - Harden the TDX "populate" ioctls against bad input, and to prepare for supporting in-place private<=>shared conversion x86 (AMD): - Forcefully invalidate SNP VMSA pages if their backing guest_memfd page is zapped/invalidated, e.g. due to a PUNCH_HOLE in response to a Page-State Change request - Remove a dying VM from the GA Log notifier list before the VM is actually destroyed, to fix a potential use-after-free - While FOLL_WRITE was needed in the past to trigger CoW unsharing, nowadays FOLL_LONGTERM does that already even without FOLL_WRITE, and in fact, get_user_pages() actually disallows FOLL_WRITE together with FOLL_LONGTERM. So don't pass FOLL_WRITE when registering encrypted memory regions, i.e. when pinning SEV/SEV-ES guest memory, to fix a regression with file-backed memory introduced by KVM's (correct) usage of long-term pins (This was reviewed by mm maintainers; for more information, see commit ee1a586dd1fa "KVM: SEV: Drop FOLL_WRITE for encrypted region registration") - Allocate full pages for SEV/SEV-ES {DE,EN}CRYPT ops on SNP-enabled hosts to fix a data corruption issue due to the PSP driver assigning to-be-written pages to firmware (as required by the SNP specs) - Unconditionally intercept ICBEP so that KVM generates the correct guest RIP when handling an ICEBP-induced TASK_SWITCH #VMEXIT - Harden the SNP "populate" ioctls against bad input, and to prepare for supporting in-place private<=>shared conversion Generic: - Remove kvm_debugfs_dir if kvm_init() fails after creating KVM's debugfs - Add a per-VM bitmap to track which vCPU IDs have been "claimed" but for which the vCPU isn't yet online, and use the bitmap to reject duplicate IDs before calling into arch code. This allows arch code to consume vcpu_id without having to worry about cross-vCPU clobbering (at least s390 and x86 have had related bugs) - Rework the so called "prepare" and "invalidate" guest_memfd hooks to prepare for in-place private<=>shared conversion, and clean up a few warts along the way Selftests: - Automatically allocate a full page for L2 guest stacks on x86 instead of requiring test-specific L1 guest code to carve out a portion of the L1 stack for L2 usage, and to ensure the L2 stack also adheres to the x86-64 calling convention ABI - Add a selftest to verify {Guest,Host}-Only behavior in x86's mediated PMU - Clean up nested SVM's handling of GPRs on L2<=>L1 transitions, reuse the functionality for nested VMX, and drop the ucall hack that was fudging around the lack of GPR switching on nVMX - Add a stress test to verify KVM doesn't clobber/drop #PF state, e.g. CR2, across save/restore, including when L2 is active - Add a test to verify KVM_CREATE_VM accepts exactly what is reported by KVM_CAP_VM_TYPES - Misc selftests fixes and cleanups - Fix several issues with seeding the pRNG, and rework the pRNG APIs to that the pRNG can be sanely used in host code, not just guest code - Add an IRQ test to validate virtual IRQ deliverty for IRQs wired up via KVM_IRQFD + KVM_SET_GSI_ROUTING, with optional support for triggering IRQs via writes to an assigned VFIO device - Add syscall wrappers to assert success on a variety of pthreads and CPU affinity APIs - Set vCPU pthread affinity as early as possible to reduce contention issues that were surfaced by PREEMPT_LAZY, which result in runtimes of over a minute on large hosts, versus the expected ~5 seconds - Rework the PMU counters test to run each testcase using a single VM with many vCPUs for each sub-testcase, instead of using a unique VM for each sub-testcase. This cuts the runtime by ~20x Miscellaneous: - MAINTAINERS updates for vfio-ap, guest_memfd, kvm-x86. Mostly representing the status quo more accurately, but also... welcome David Hildenbrand as guest_memfd reviewer!" * tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (413 commits) KVM: arm64: Validate GICv5 timer PPIs before claiming ownership KVM: arm64: vgic: Reject out-of-range GICv5 PPI IDs KVM: arm64: vgic: Prevent speculative SPI array underflow KVM: arm64: vgic: Free gic_kvm_info on initialization failure KVM: arm64: Avoid mismatched accesses to 'struct kvm_nvhe_init_params' s390/vfio-ap: Fix NULL deref in status_show() during queue probe s390/vfio-ap: Fix hot-unplug skipped when last AP adapter or domain removed s390/vfio-ap: fix potential use of uninitialized apm_filtered bitmap s390/vfio-ap: Fix control domain removal in vfio_ap_mdev_cfg_remove s390/vfio-ap: Fix required lock not held during update of ap_matrix_mdev object s390/vfio-ap: Fix missing lock required to access list of ap_matrix_mdev objects s390/vfio-ap: Fix dereference matrix_mdev->kvm without checking for NULL s390/vfio-ap: Fix stale do_remove flag across iterations in vfio_ap_mdev_cfg_remove RISC-V: KVM: fix vcpu vector context handling for kernel-mode vector riscv: vector: allow non-preemptible kernel-mode vector with IRQs off riscv: vector: refactor riscv_v_start_kernel_context KVM: s390: gmap: Make prefix handling optional KVM: s390: gmap: Make CMMA optional KVM: s390: gmap: Make storage keys optional KVM: s390: Prepare gmap for a second KVM implementation ...

Merge tag 'kvm-x86-misc-7.3' of https://github.com/kvm-x86/linux into HEAD

2026-08-18T11:41:51Z

KVM x86 misc changes for 7.3 - Fix VPID virtualization bugs where KVM would fail to flush hardware TLBs. - Harden the SNP and TDX "populate" ioctls against bad input, and to prepare for supporting in-place private<=>shared conversion. - Fix a variety of #DB priority bugs. - Fix a class of races related to enabling Hyper-V emulation on a vCPU after the vCPU is visible to the rest of KVM. - Use static calls for nested virtualization ops. - Move more KVM-internal code out of x86's kvm_host.h. - Enumerate support for a variety of Zhaoxin instructions that don't require explicit virtualization. - Fix missing EFER validation bugs, including in the KVM_SET_SREGS* path. - Harden kvm_vcpu_map() against double-mapping and thus leaking references. - Misc fixes and cleanups, e.g. for largely benign syzkaller splats.