| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Pull RDMA updates from Jason Gunthorpe:
"About the normal size, still a lot of AI bug fixes and so on, but some
interesting new functionality too:
- Assorted locking, bounds-checking, cleanup, and error-path fixes
across UCMA/CMA, bng_re, bnxt_re, cxgb4, EFA, ERDMA, HFI1, HNS,
ionic, iRDMA, mlx4/mlx5, RXE, SIW, SRP/SRPT, and iSER target.
- netlink report for max # of supported resources
- get_zeroed_page()/etc removal
- Robust udata for ionic
- Allow unique RDMA device names per network namespace
- Completion counters and v2 admit queue support for EFA
- UC QP support for MANA
- Completion timestamps for ionic
- Harden uverbs data validation and resource lifetime handling,
fixing several core use-after-free conditions.
- bnxt_re toggle-page ownership and lifetime bug fixes
- dmabuf SRQ support for mlx5"
* tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma: (160 commits)
RDMA/ucma: Allow path records to exactly fit the output buffer
RDMA/uverbs: Guard legacy bundles without method_elm
RDMA/efa: Add support for 128B admin v2 SQ entry
RDMA/efa: Generalize the admin SQ
RDMA/efa: Decouple admin command payload from admin header
RDMA/rxe: Fix OOB in free_rd_atomic_resources()
RDMA/cma: Fix WARNING in res_to_rt
RDMA/cxgb4: Free debugfs on registration failure
RDMA/cxgb4: Cancel reg_work before freeing device on remove
RDMA/ucma: Lock the handler in ucma_set_ib_path()
RDMA/ucma: Lock the handler in ucma_write_cm_event()
RDMA/erdma: restrict the driver to little-endian systems
RDMA/ionic: Embed counter driver data in rdma_counter allocation
RDMA/ionic: Cap eq_count to the eth driver's interrupt vector budget
RDMA/siw: Fix use-after-free in siw_accept()
IB/isert: post the full-feature receive buffers after session registration
IB/isert: delay the final Login Response until the session is registered
RDMA/srp: fix heap information leak on a truncated SRP_CRED_REQ
RDMA/erdma: Hold QP references for AE and CM processing
RDMA/erdma: Hold CQ references when processing EQ events
...
|
|
Pull drm updates from Dave Airlie:
"Highlights:
- dmemcg eviction support is good for low VRAM things like Steam
Machine
- AMD adds gfx6-8 modifier support for older GPUs that enables a
bunch of wayland stuff
- i915/xe has some new hw support but also a lot of display
refactoring
Everything:
perf:
- export perf_allow_ APIs for xe
udmabuf:
- remove default size limit of 64MB
rust:
- i/o rework (signed tag from driver-core tree)
- add registration guard and registration data
- fix unbounded lifetimes in ioctl handler args
- fix a drm_dev_register race
- gem_shmem: add DmaResvGuard helper
- gpuvm: require send/sync for driver data
- implement send/sync for GpuVaAlloc and GpuVmBo
- add SmContext lifetime
- rename dma_handle to dma_address
- change pci_sriov_get_totalvfs return to unsigned int
core:
- create drm_of_get_panel_orientation
- send per-connector hotplug events
- add thunderbolt UBHR tunneling support
connector:
- add color format property
dmem:
- introduce a peak file
- accept one region per limit
- add dmemcg support for eviction
gpusvm:
- reorg code to give drivers more flexibility
atomic:
- add create_state callback and helper
- add documentation on atomic commit lifetime
buddy:
- add per-order free
- add used block scoreboard
- fix UAF
- test buffer clearance on resume
- add phys_addr->block helper
gem:
- drop DRIVER_GEM_GPUVA flag
ttm:
- be more aggressive allocating below protection limit
sched:
- add test suite for concurrent job submissions
hdmi:
- hook the color format property in helpers
mipi-dsi:
- add MIPI_DSI_MODE_DSC_ALL_SLICES_IN_PKT
bridge:
- add atomic create callbacks
- drop atomic reset
- display-connector: don't autoenable HPD IRQ
- trigger initial HPD for DP
- ti-sn65dsi83: remove NO_HFP and NO_HBP mode flags
- analogix_dp: switch to DP link training helpers
dp:
- add support for DSC max delta BPP
edid:
- parse panel type from DisplayID 2.x Display Parameters
sysfb:
- improve panel, stride, framebuffer size validation
panel:
- implement ref counting for struct drm_panel
- himax-hx83121a: add backlight regulator support
- novatek-nt36672a: Inline panel init sequences
- visionox-vtdr6130: enable DSC
- novatek-nt37801: Use mipi_dsi_*_multi() functions
- samsung-s6d16d0: Fix prepare error handling
- support Novatek NT36536 plus DT bindings
- sofef00: fix backlight updates
- osd101t2587: use mipi_dsi_*_multi interface
- panel-edp: adjust timing for AUO displays
- panel-lvds: support Opto Logic SCX1001511GGC49
- panel-simple: support Kyocera tcg070wvlq
- panel-edp: quirks
- AUO B116XAT04.3, CMN N116BCP-EA2, CSW MNB601LS1-8
- BOE NV116WH2-M30, BOE NT116WHM-N21, BOE NV116FH1-M31
- BOE NV116FH1-M30, NV140FHM-N5B, TM156VDXP25
- BOE NE160QDM-NY1, MB116AS01
- new:
- Samsung ATNA40HQ08-0, Anbernic TD4310
- Chipone ICNA35XX, Ilitek ILI9488
- Ilitek ILI7807S, Renesas R63419
- MNE001BS6-2, MNF601BS4-1, Sharp LQ120P1JX51
virtio:
- add support for save/restore virtio_gpu_objects
- abort vq wait on device removal
amdgpu:
- add color format DRM property
- initial compute pipe reset support
- add GFX 6-8 modifier support
- initial DCN 6.0.0 support
- dmemcg eviction support
- improved boundary checking for bios parsing
- RAS updates and rework
- VCN secure submission fixes
- 8K panel fix
- Display KUNIT tests
- parse panel type from DisplayID
- Align IP discovery to pci device lifetime
- SOC15 register macro cleanups
- UVD memory placement fixes
- GFX9 mode2 reset fixes
- drop unnecessary BUG/BUG_ON
- GFX8 soft reset rework
- enable soft reset on GFX8
- PSP/SMU 15.0.9 update
- VI ASPM fix
- userq fixes
- amdgpu_vm_get_task_info_pasid lifetime fix
- DC CACP support
- change system_unbound_wq with system_dfl_wq
- Loosen VFCT bios parsing to deal with pci=realloc
- SI/SMU7 AC/DC switch fix
- VM fence handling fix
- GEM close optimisation
- Apple Studio Display fixes
- DC FRL fixes
amdkfd:
- initial compute pipe reset support
- allow applications to opt out of sigbus on fatal errors
- improve CRIU boundary checks
- MQD handling rework
- move TBA/TMA from system to device memory
- avoid topology-lock in kfd_mmap
- SVM eviction fixes
radeon:
- fix unset CONFIG_ACPI build
i915:
- Novalake (NVL display version 35) timing generator enabling
- NVL DC3CO enabling
- enable UBHR link rates on thunderbolt tunnels
- Reduce Xe3+ PM demand peak bandwidth
- enable pipe DMC error interrupts for display 30+
- add kunit tests for DP link config selection
- refactor and document DP link recovery
- i915/xe driver display probe/remove/suspend/resume/shutdown cleanup
and unification
- i915/xe display runtime PM unified
- Break i915 and xe panic dependency on struct intel_framebuffer
- Streamline Pre/Post-CSC LUT loops
- drop TGL DC3DO support
- CDCLK santization
- fix HDMI scrambling enable
- fix phys bo pread/pwrite with offset
- add missing nospec on parallel submit slot
- fix some NULL derefs
xe:
- drop force_execlist module param
- gate observation streams with perf_allow_cpu
- skip FORCE_WC and vm_bound check for external dma-bufs
- dmemcg eviction support
- remove unused NVL-S GuC
- TLB invalidation improvements
- NVL-S updated PCI-IDs and w/a
- madvise: optimise invalidation path
- fix infinite gt-reset loop in timeout recovery
- update TTM device benefical_order
- wait on external BO kernel fences in exec ioctl
- add/use more KLV helpers
- sriov: disable display in admin only PF mode
- add RAS GPU health indicator
- optimise TTM populate for DONTNEED BO
- drop force_probe for NVL-s
- add debugfs for pcode info
amdxdna:
- disable device buffer export
nova:
- build nova-core/nova-drm from drivers/gpu
- export nova-core rust symbols (workaround)
- GSP boot process consolidation
- Boot GSP with vGPU enabled
- TLV firmware image format support
- Hopper/Blackwell fixes and cleanups
- I/O projection adoption
tyr:
- firmware loading and MCU boot
- add generic slot manager + MMU
- GPU VM support ARM64 LPAE page tables
- add kernel buffer object for internal allocations
- add parser for Mali CSF
- add MCU booting
nouveau:
- race fixes
- check instmem iomapping at first use
- add dmemcg support
- expose NVDEC channels
- add scanline position/head state support for GSP
qxl:
- convert simple encoder to regular
ethosu:
- add perf counter support
etnaviv:
- force flush on power register ops
msm:
- support DSC configuration with slice_per_pkt > 1
mxsfb:
- fix disable sequence
panthor:
- support sparse mappings
rockchip:
- switch away from simple helpers
- support YUV background color
- fix layer config timeout
- add edp support for rk3576
- add batch command submission function
rocket:
- error handling and NULL ptr deref fixes
sun4i:
- switch away from simple helpers
imagination:
- mark BXM-4-64 MC1 as support
host1x:
- support tegra264
tegra:
- add DSI for tegra 20/30
v3d:
- reduce PM runtime autosuspend delay
- scheduler fixes and refactoring
- deprecate v3d 3.3 and 4.1
- validate CPU job query boundaries
hibmc:
- improve plane format handling
- switch to gem shmem
mediatek:
- cec: correct compat for mt7623-8167?
exynos:
- remove simple dependency
- add error handling to encoder paths
- take i2c adapter module reference"
* tag 'drm-next-2026-08-20' of https://gitlab.freedesktop.org/drm/kernel: (2074 commits)
drm/xe/mcr: Take vcs1/vecs1 into account for first media slice
drm/xe: Fix a bug in pc_adjust_freq_bounds()
drm/xe: Fix xe_device_probe() failure
drm/xe/drm_ras: Move has_drm_ras check to drm_ras layer
drm/xe/ras: Fix boot-time ras error processing
drm/amd/display: make DC_RUN_WITH_PREEMPTION_ENABLED misuse a build error
drm/amd/pm: silence uninitialized variable warnings
drm/amdgpu: skip BOs being torn down during GTT recovery
drm/amdgpu: Reject UVD message with invalid number of h265 refs
drm/amdgpu: keep PRT mappings off the vm_bo state lists
drm/amdgpu: fix nbif 6.3.1 l1 low power not functional
drm/amd/display: fix BT.2020 YCbCr output CSC matrices for DCE
drm/amd/display: fix BT.2020 YCbCr limited output CSC matrix
drm/amdgpu: Implement insert_end for VCE 3
drm/amdgpu: Fix UVD min buffer sizes
drm/amdgpu: Fix UVD decode image min size calculation
drm/amdgpu: Fix UVD dpb min size calculation for H264
drm/amdgpu: Reject UVD message with dimensions above 4096
drm/amdgpu: check ASPM on the dGPU host link
drm/radeon: fix autosuspend cleanup during teardown
...
|
|
Pull MM updates from Andrew Morton:
- "mm: drop "sub" prefix from various places" (Dev Jain)
page->folio conversion and a naming cleanup
- "mm/kasan: remove redundant initialization for kasan_flag_write_only"
(Igor Putko)
KASAN cleanup work
- "mm/filemap: reduce unnecessary xarray lookups" (Chi Zhiling)
Small speedup in the pagecaache read code
- "mm/percpu: Fix possible NOFS/NOIO reclaim recursion" (Kaitao Cheng)
Improve the vmalloc code - mainly the avoidance of GFP_KERNEL
allocations when the caller asked for GFP_NOFS or GFP_NOIO
- "mm/kmemleak: avoid soft lockup when scanning task stacks" (Breno
Leitao)
Avoid a soft lockup watchdog trigger from the kmemleak scanning code
in extreme situations
- "mm/page_owner: misc cleanups" (Ye Liu)
Cleanups to the page_owner code. For some reason lots of people have
been working on the page_owner code this cycle.
- "mm: convert to walk_page_range_vma() to eliminate find_vma()"
(Kefeng Wang)
Simplify and accelerate the page walking library function
- "mm/migrate: preparatory cleanups for batch copy and offload"
(Shivank Garg)
Cleanups in the migration code
- "mm/page_owner: add per-fd filter infrastructure for print_mode and
NUMA filtering" (Zhen Ni)
Per-fd filtering to page_owner in order to reduce the sometimes vast
amount of output it can produce
- "mm: Refactor bootmem gigantic hugepage allocation" (Muchun Song)
Fixes and preparatory cleanups around bootmem HugeTLB handling,
sparse initialization ordering, and related vmemmap setup
- "mm/zsmalloc: reduce lock contention in zs_free()" (Wenchao Hao)
Reduce lock contention in zs_free(), which dominates the unmap path
under memory pressure on Android (LMK kills) and on x86 servers
running zswap-heavy workloads.
Up to 1.83x improvement in microbenchmarking.
- "move alloc_tag.c file under mm/" (Suren Baghdasaryan)
- "samples/damon: handle damon_{start,stop}() failures" (SJ Park)
Fix improper handling of damon_start(), damon_stop(), and
damon_call() failures across DAMON sample modules to prevent
potential memory leaks, operation disruptions and use-after-free
bugs
- "mm/damon/sysfs: kobject_del() directories that users can
create/remove" (SJ Park)
Fix delayed sysfs directory removal under DEBUG_KOBJECT_RELEASE
causeing creation failures due to duplicate directory names by adding
missing kobject_del() calls before creating new directories
- "mm: cleanup clear_not_present_full_ptes()" (David Hildenbrand)
Clean up the core pte handling code
- "selftests/damon: misc fixes for test bugs" (Kunwu Chan)
Fix several bugs in the DAMON selftests
- "selftests/damon: fix memcg_path staging handling" (Cheng Nie)
Fix a bug in _damon_sysfs.py for damos_filter memcg_path setup, and
add a test case for it in sysfs.py.
- "selftests/damon: test kdamond refresh_ms" (Ruslan Valiyev)
Selftest coverage for DAMON's refresh_ms sysfs feature by updating
the test control module and verifying that scheme stats update
automatically without manual intervention
- "mm/damon: five misc fixups" (Akinobu Mita)
Miscellaneous DAMON fixups.
- "mm/damon/core: detect internal variation above max_nr_regions/2"
(Jiayuan Chen)
Fix DAMON's region splitting behavior when region counts exceed half
the maximum budget by dynamically scaling down the split fraction as
the limit approaches, preventing large regions from staying un-split,
and add corresponding KUnit test coverage
- "mm: preparatory patches for PMD level swap entries" (Usama Arif)
Refactor and clean up PMD softleaf helpers, call sites, and
architecture flags to lay the groundwork for a follow-up series that
introduces PMD page table swap entries
- "mm/damon: update, optimize, and clean up doc, tests, and code" (SJ
Park)
Update DAMON design and ABI documentation, expands unit and selftest
coverage, optimize damon_commit_target_regions(), and clean up
recently added sysfs interface code for better readability
- "mm/vmpressure: reduce CPU, memory and code overhead on cgroup v2"
(Usama Arif)
Optimize vmpressure() by skipping unnecessary work on cgroup v2 for
userspace event notifications and refactor v1-only eventfd handling
into mm/memcontrol-v1.c to reduce memory overhead and code complexity
- "selftests/mm: refactor pkey helpers and fix mmap error handling"
(Hongfu Li)
Refactor pkeys shared tracing and assertion helpers into a common
file, unify protection key selftests to use consistent diagnostic
logging and assertions, and enforce standardized MAP_FAILED return
checks for mmap() calls across the tests
- "mm/damon: optimize out nr_accesses_bp" (SJ Park)
Replace the error-prone, continuously updated nr_accesses_bp field in
damon_region with an on-demand moving sum function, reducing
structure memory overhead and avoiding state corruption bugs
- "Open HugeTLB allocation routine for more generic use" (Ackerley Tng)
Decouple HugeTLB folio allocation from VMA dependencies by
introducing hugetlb_alloc_folio(), enabling subsystems like
guest_memfd to allocate HugeTLB folios without standard VMA
reservations or pseudo-VMAs
- "mm/damon: provide pseudo moving sum probe_hits" (SJ Park)
Integrate DAMON's probe_hits attribute counter into the pseudo moving
sum infrastructure, enabling real-time, online monitoring without
waiting for full aggregation intervals
- "mm: Some cleanups for page allocator APIs" (Brendan Jackman)
Simplify and refactor the page allocator entry points and flags by
unifying allocation paths, adding internal alloc_flags arguments, and
eliminating redundant __ prefixed alloc_pages variants.
- "Fix incorrect access of hugetlb pte entries" (Dev Jain)
Enforce the consistent use of huge_ptep_get() instead of ptep_get()
for HugeTLB entries and fixes an unaligned address issue in arm64's
huge_ptep_get() implementation
- "mm/damon: validate all parameters in the core" (SJ Park)
Consolidate parameter validation into the DAMON core specifically
within damon_start() and damon_commit_ctx() to centralize error
checking, eliminate caller-side redundant checks and to improve
maintenance efficiency
- "tools/mm/page_owner_sort: fix filtering and cleanup issues" (Yichong
Chen)
Rename is_need() to filter_record() for clearer return semantics, fix
per-record allocation memory leaks and bound output copies in
search_pattern() to address an existing buffer issue
- "memcg: bail out reclaim when memcg is dying" (Jiayuan Chen)
Mitigate a system-wide stall which occurs when a cgroup is removed
while one of its memory control files is doing synchronous reclaim
- "mm/memory-failure: add panic option for unrecoverable pages" (Breno
Leitao)
Introduce an opt-in vm.panic_on_unrecoverable_memory_failure sysctl
that immediately panics the kernel on unrecoverable memory errors in
kernel-owned pages to preserve error context and prevent delayed,
silent data corruption
- "mm/damon: refactor damon_{start,stop,commit}() for simple error
handling" (SJ Park)
Refactor the DAMON core API functions to guarantee that all contexts
are fully stopped when damon_start(), damon_stop(), or damon_commit()
fail, eliminating the need for complex and error-prone caller-side
cleanup code
- "Keep tail page private zero at free and folio split" (Zi Yan)
Add checks to ensure tail_page->private is zero when freeing compound
or high-order pages and when promoting tail pages during large folio
splits. By validating these fields at free and split time, it allows
the removal of redundant private field clearing inside
prep_compound_tail()
- "mm: drop redundant lru_add_drain in anon folio reuse paths" (Barry
Song)
Eliminate redundant lru_add_drain() calls in
wp_can_reuse_anon_folio() and do_swap_page() to reduce LRU lock
contention and system overhead
By validating folio refcounts against the LRU cache before draining
and removing unnecessary drains in the swap path, it achieves up to a
30.5% reduction in drain calls during heavy swap workloads
- "mm: clean up folio LRU and swap declarations" (Jianyue Wu)
Reorganize folio LRU and swap code by relocating page-cluster state
to mm/swap_state.c, renaming mm/swap.c to mm/folio.c, and moving
MM-internal reclaim declarations into mm/internal.h.
- "userfaultfd: working set tracking for VM guest memory" (Kiryl
Shutsemau)
Add userfaultfd support for tracking the working set of VM guest
memory, so a VMM can identify hot pages and reclaim cold ones to
tiered or remote storage
- "mm: remove CONFIG_HAVE_BOOTMEM_INFO_NODE (Part 2)" (David
Hildenbrand)
Remove the remaining pieces of CONFIG_HAVE_BOOTMEM_INFO_NODE,
performing some smaller cleanups around freeing of reserved vmemmap
pages on the way.
- "mm/damon: update probe hits for runtime parameter commits" (SJ Park)
Ensure that DAMON's probe_hits attribute counter is properly updated
when monitoring intervals are changed at runtime, matching the
behavior of nr_accesses. To achieve this, it refactors and renames
existing helper functions for shared use, applies the updates to
probe_hits, and handles edge cases in damon_probe_hits_mvsum() to
maintain measurement accuracy.
- "KSM: performance optimizations for rmap_walk_ksm" (xu xin)
Resolve a severe KSM reverse-mapping performance bottleneck where
thousands of split VMAs sharing a single anon_vma cause extended lock
contention.
By adding an interval-filtering check during the rmap walk, it
reduces worst-case anon_vma lock hold times from over 500ms down to
under 2ms, preventing application freezes and latency spikes under
memory pressure.
- "mm: split a couple of headers from internal.h" (Mike Rapoport)
Split declarations related to mm_init, memblock, vmalloc and sparse
into new headers
- "KSM: use linear_page_index in collect_procs_ksm()" (xu xin)
Apply the interval tree optimization from rmap_walk_ksm() to
collect_procs_ksm() to avoid iterating over non-matching VMAs during
KSM memory error handling.
It hoists loop-invariant address initialization and restricts the
anon_vma_interval_tree_foreach walk to a targeted page offset range,
reducing redundant checks and improving lookup efficiency.
- "selftests/mm: avoid false failures in hugetlb and KSM tests" (Sayali
Patil)
Fix issues in the hugetlb and KSM MM selftest categories that can
report failures when the prerequisites for the tests are not
satisfied
- "mm/damon: introduce data attributes only monitoring" (SJ Park)
Introduce attribute-weighted region management in DAMON, allowing
users to prioritize specific data attributes (such as page sizes or
cgroups) over or instead of access monitoring.
By assigning weights to attribute probes, DAMON can completely
disable access tracking and adjust monitoring regions based on
weighted probe-hit counters to optimize monitoring quality for
attribute-focused workloads.
- "mm/hmm: Add mmap lock-drop support for userfaultfd-backed mappings"
(Stanislav Kinsburskii)
Extend hmm_range_fault() to support userfaultfd-backed regions by
allowing the mmap lock to be dropped during fault handling via a new
hmm_range_fault_locked() helper.
By accepting a locked pointer and signaling retry status when lock
release occurs, it enables page fault resolution in userfaultfd
regions while preserving backward compatibility for existing callers.
- "mm: make VMA page offset handling more consistent" (Lorenzo Stoakes)
Clean up and standardize how vma->vm_pgoff is accessed and
manipulated across file-backed and anonymous mappings in the kernel
It introduces dedicated helper functions such as vma_start_pgoff(),
vma_end_pgoff(), vma_set_pgoff() and linear_page_delta() while
renaming rmap interval tree helpers to better reflect their
functionality.
These changes establish a cleaner foundation for future work that
will unify virtual page offset indexing for all anonymous and CoW'd
folios.
- "mm: handle device-private PMDs in walk callbacks" (Usama Arif)
Address kernel panics and state corruption caused by MM walk
callbacks reaching non-present device-private PMD swap entries
created during HMM migrations
It ensures that functions which acquire pmd_trans_huge_lock()
properly recognize device-private PMDs instead of assuming a present
THP or a standard migration entry.
- "mm/rmap: Refactor try_to_unmap_one" (Dev Jain)
Refactor try_to_unmap_one by modularizing Hugetlb,
anonymous-lazyfree, and anonymous-swapbacked logic into dedicated
functions, laying the structural groundwork for batched anonymous
large folio unmapping.
- "Docs/ABI/damon: sysfs ABI document fixes and additions" (Song Hu)
Fix typos and fills in missing entries in the DAMON sysfs ABI
document
- "dax/kmem: atomic whole-device hotplug via sysfs" (Gregory Price)
Introduce an atomic sysfs state attribute and supporting DAX/MM
infrastructure to prevent userland races when offlining and removing
entire memory regions
By adding an unplugged state alongside standard online modes, it
enables whole-device atomic hotplug control while preserving backward
compatibility.
- "mm: convert more vm_flags_t users to vma_flags_t" (Lorenzo Stoakes)
Continue transitioning the kernel from the deprecated vm_flags_t type
to vma_flags_t across core memory management infrastructure.
It replaces legacy type usage in core functions such as do_mmap(),
unmapped area allocation, mm->def_vma_flags, and VMA operations like
mlock, mprotect, and mremap.
- "Two small patches to clean up mm/mm_slot.h" (xu xin)
Refactor mm_slot.h by introducing mm_slot_remove() to unify duplicate
slot deletion sequences in khugepaged and KSM. It also adds code
documentation explaining why mm_slot_lookup and mm_slot_insert must
remain as preprocessor macros rather than static inline functions.
- "mm/damon/core: hide core-private struct fields" (SJ Park)
Clean up DAMON core structures by consistently marking internal-only
fields with private: comment tags to prevent improper direct access
from outer layers.
It enforces encapsulation across core structures including
damon_region, damon_target, and damon_ctx and updates DAMON_SYSFS to
interact through approved access APIs instead of exposing raw struct
members.
- "mm/damon: unurgent fixes for infinite loop, NULL de-ref and races"
(SJ Park)
Address potential infinite loops, NULL dereferences, and race
conditions identified in DAMON
It fixes an infinite loop triggered by extreme user configurations, a
NULL pointer dereference within unit tests and minor monitoring
accuracy degradation caused by subtle runtime races.
- "mm/page_alloc: fixes for free_pages_nolock() on RT/UP" (Brendan
Jackman)
Fix an NMI safety flaw in __free_frozen_pages() where freeing pages
on non-SMP or PREEMPT_RT kernels can bypass can_spin_trylock() checks
via non-PCP or isolated migration paths.
It also resolves potential kernel crashes and privilege escalation
risks triggered when BPF tracing runs in NMI context alongside memory
hotplug or large allocation frees.
- "mm/page_alloc: couple of followups for recent cleanups" (Brendan
Jackman)
Clean up and update page allocator nomenclature, documentation, and
debug assertions.
It aligns internal FPI_ flags with the public "nolock" naming
convention, removes outdated internal implementation details from
high-level page allocator comments, and eliminates obsolete
VM_BUG_ON() assertions in allocation paths.
- "mm/mseal: further cleanups" (Lorenzo Stoakes)
Refactor and simplify the mseal implementation by clarifying API
boundaries and removing unnecessary code complexity.
It replaces generic do_mseal() usage outside the syscall with a
dedicated mseal_mmap_page_zero() helper for MMAP_PAGE_ZERO,
eliminates mm_struct parameters to enforce that sealing applies only
to current->mm, and streamlines overall logic and comments with no
functional changes intended.
- "mm/vmscan: fix swappiness=max and clean up per-node proactive
reclaim" (Ridong Chen)
Resolve reclaim behavior bugs and clean up function parameters across
memory reclaim paths
It fixes swappiness=max in both standard reclaim and MGLRU so
unswappable anonymous memory no longer falls back to evicting page
cache, ensures reclaim_store() returns accurate error codes instead
of collapsing all failures into -EAGAIN, and removes the obsolete
gfp_mask parameter from __node_reclaim().
- "mm: mincore: misc cleanups" (Kefeng Wang)
Clean up and simplifies the mincore code. Most importantly, it
removes the historical special behavior that always reports VM_PFNMAP
pages as non-resident.
- "mm/huge_memory: drop dead split helper variants" (Kiryl Shutsemau)
Two trivial cleanups in the folio split API
- "mm/damon: fix uninitialized DAMOS field and kunit exec expectation
bugs" (SJ Park)
Resolve minor operational and testing bugs in DAMON identified by
Sashiko. It initializes the damos->last_applied field to prevent
occasional efficiency degradation and fixes invalid memory accesses
in DAMON KUnit tests during test failure handling.
- "cleanup for stable_page_flags()" (Jinjiang Tu)
Clean up and refactor stable_page_flags() used by /proc/kpageflags
without altering functionality.
It uses BIT_ULL() to prevent shift-overflow warnings on 64-bit flag
bits, converts folio-specific flag checks to standard folio_test_*()
helpers, and removes redundant CONFIG_PAGE_IDLE_FLAG handling.
- "Batch unmap of uffd-wp file folios" (Dev Jain)
Extend batched folio unmapping support to file folios within
userfaultfd write-protect (uffd-wp) VMAs by adding batching
capabilities to pte_install_uffd_wp_if_needed().
This removes special-case restrictions on uffd-wp VMAs in
try_to_unmap_one(), significantly simplifying the function's control
flow and complexity.
- "mm/early_ioremap: clarify and clean up early_ioremap_reset()"
(Sang-Heon Jeon)
Clarify and clean up the architecture-specific usage of
__late_set_fixmap() and __late_clear_fixmap() after
early_ioremap_reset()
It adds explicit documentation regarding when early_ioremap_reset()
must be called and removes redundant macro definitions and reset
calls in the RISC-V and ARM64 architectures.
- "mm: fix reclaim storms in defrag_mode" (Johannes Weiner)
Address severe performance regressions, swap storms, and spurious
OOMs caused by vm.defrag_mode=1 under high memory pressure in Meta
production
It updates the page allocator slowpath so non-movable allocation
requests actively trigger direct reclaim and direct compaction at
pageblock_order scale, allowing them to claim whole pageblocks rather
than spinning unproductively.
- "zram: lockmap tweaks" (Sebastian Siewior)
Optimize and fix lockdep tracking for zram devices by consolidating
per-entry lockmaps and isolate lock classes across multiple instances
This reduces memory overhead by replacing per-entry lockdep_map
instances with a single map per struct zram, and assigns a dynamic
lock_class_key to each instance to prevent false deadlock reports
when different zram devices are backed by distinct filesystems.
* tag 'mm-stable-2026-08-18-18-39' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (501 commits)
selftests/mm: thuge-gen: fix test_shmget() for PAGE_SIZE check
selftests/mm: unpoison pages in memory-failure teardown
mm/shmem: downgrade final i_blocks check in shmem_evict_inode() to pr_warn()
mm/khugepaged: replace mutex_lock/mutex_unlock usage with guard macro
mm/zsmalloc: fix release order of locks in zs_page_migrate()
Documentation: zram: remove sections numbering
ksm: stop iterating VMAs when ksm_test_exit returns true
mm: fold userfaultfd_rwp() to false without CONFIG_ARCH_HAS_PTE_PROTNONE
mm/migrate: report RCU-tasks quiescent states in migrate_pages_batch()
zram: use a custom key for each zram object
zram: move lockmap to be per-zram instead per table
selftests/mm: fix gup_longterm EINVAL error message
mm: page_alloc: fix non-movable reclaim storm in defrag_mode
mm: page_alloc: move capture_control to the page allocator
mm: compaction: support non-movable compaction for pageblock requests
mm: page_alloc: __GFP_FS lockdep annotation for direct compaction
hugetlb: evaluate subpool free state while locked
mm/damon: remove trailing semicolons after function definitions
mm/damon/ops-common: prevent migration fallback to non-target nodes
mm/damon: update outdated comment about DAMOS filter handling
...
|
|
Pull sched_ext updates from Tejun Heo:
"Most of this cycle completes the enqueue-path support for hierarchical
sub-scheduling, which makes sub-scheduler support feature complete: a
root BPF scheduler can now hand a cgroup subtree over to a nested
sub-scheduler together with revocable CPU grants, and the
sub-scheduler owns all scheduling decisions for its tasks on those
CPUs.
Development volume was high and a number of changes plugging holes in
the new support landed late in the cycle. Also included are core
scheduling fixes that were completed too late for the v7.2 release and
are routed through this pull request.
Sub-scheduler CPU delegation:
- Parent schedulers now grant and revoke per-CPU capabilities
(enqueueing, preemption, CPU frequency control) on their children,
enforced on every path a scheduler can reach a CPU through.
Previously only dispatching could be delegated; this lets
sub-schedulers fully schedule their CPUs.
- Rescue execution: a task whose scheduler doesn't have access to the
CPUs the task needs to run on starved until the watchdog ejected
the whole scheduler. The kernel now runs such tasks directly on a
small bandwidth budget, turning a scheduler-killing failure into
bounded degradation.
- Cgroup integration: tasks migrating across a sub-scheduler boundary
weren't re-homed to the new owner, causing wrong-scheduler
scheduling and a use-after-free. Sub-schedulers now take over their
cgroup subtree and receive its cgroup callbacks.
- Arena objects now cross the kernel/BPF boundary as typed pointer
arguments, translated transparently by the BPF tree's new arena
argument support, replacing untyped arguments with manual
translation.
- scx_qmap now demonstrates full hierarchical sub-scheduling.
Other fixes and updates:
- Robustness improvements: the abort path is now NMI-safe, fixing
deadlocks when errors are raised from NMI context and making
hardlockup recovery direct. Reenqueue loops that could monopolize a
CPU ahead of the watchdog now eject the offending scheduler, and
stalls are blamed on the scheduler actually responsible.
- Hardening: BPF-writable arena memory is validated before kernel
use, and task slice and vtime writes got explicit synchronization
rules, closing corruption vectors open to buggy or malicious
schedulers.
- Core scheduling: sched_ext dispatching can drop the rq lock inside
the core-wide pick, which let interleaving selections corrupt each
other's state and hard-hang the machine. The selection now restarts
when the lock was released. The task ordering callback was also
invoked with its arguments swapped, and the default ordering is
updated to work across sub-scheduler boundaries. The fixes are
marked for stable.
- Other fixes headed for stable: a task init leak on fork failure
during enable, tooling compat macros that silently failed to detect
newer kernels, and a crash on reenqueueing against a destroyed
dispatch queue.
- Tooling: scx_pair moves off deprecated callbacks, and the
deprecated scx_bpf_cpu_rq() kfunc is removed"
* tag 'sched_ext-for-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext: (144 commits)
sched_ext: Drop the dead SCX_DEQ_CORE_SCHED_EXEC test in dequeue_task_scx()
sched_ext: Make core-sched task ordering hierarchy-aware
sched_ext: Use runnable_at for the default core-sched task ordering
sched_ext: Fix inverted ops.core_sched_before() invocation
sched_ext: Move the config-off sub-cap kfunc stubs into sub.c
sched_ext: Rename balance-era identifiers to dispatch terms
sched_ext: Drop the stale keep_prev fixup in dispatch_pick()
sched_ext: Keep kick_sync waiting on the rq's own CPU
sched_ext: Make SCHED_CLASS_EXT select GENERIC_ALLOCATOR
sched_ext/scx_flatcg: Fix cvtime true-up on slice expiry
sched_ext: Don't BUG_ON a destroyed DSQ in process_deferred_reenq_users
sched_ext: Fix scx_bpf_dsq_move_to_local___v2 compat detection
sched_ext: Make scx_bpf_events() read the calling scheduler's counters
sched_ext: Drop unlocked scx_rq_clock_invalidate() from scx_root_disable()
selftests/sched_ext: Fix flaky ddsp failure tests on busy systems
selftests/sched_ext: Make numa idle validation race-free
sched_ext: Fix scx_bpf_dsq_reenq___compat kfunc extern prototype
sched_ext/scx_flatcg: expire cached hweights on weight changes
sched_ext: Fix exit_task leak on fork failure during enable
sched_ext: fix stale references in doc comments
...
|
|
Pull cgroup updates from Tejun Heo:
- Attach path bug fixes: migrations spanning multiple source or
destination cpusets were mishandled, most visibly leaving thread
affinities stale when the controller is disabled in a threaded
subtree. Configuration writes could also race an in-flight attach and
apply stale state, and the deadline task count could get corrupted by
concurrent updates, skewing SCHED_DEADLINE admission decisions.
- Memory binding bug fixes: which node masks get applied differed
between the binding update paths, and tasks cloned with
CLONE_INTO_CGROUP skipped rebinding entirely. Rebinding also now runs
once per process instead of repeating for every thread sharing the
mm.
- Overhead removals with no behavior change: CPU hotplug iterated tasks
of cpusets that just inherit the parent's effective masks, and the
slab-spreading task flag was still being maintained although the SLAB
allocator that consumed it is long gone.
- Data-race annotations for benign races so that KCSAN reports stay
meaningful, selftest coverage for the fixes above along with
flakiness and portability fixes, and documentation corrections.
* tag 'cgroup-for-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup: (34 commits)
selftests/cgroup: Remove redundant chown in test_cgcore_lesser_ns_open
selftests/cgroup: Preserve CPU hotplug write errors
cgroup/cpuset: Add test for partition root invalidation returning wrong CPUs
cgroup/cpuset: Remove obsolete PFA_SPREAD_SLAB task flag
docs: cgroup-v2: fix stale "io" controller introduction
selftests/cgroup: Avoid awk -e in cpuset tests
cgroup/cpuset: Use WRITE_ONCE() for shared prs_err updates
selftests/cgroup: add user_usec sanity check in test_cpucg_nice
cgroup: drop unneeded semicolon
docs: cgroup-v2: mark memory.pressure and io.pressure as read-write
selftests/cgroup: Fix minor defects in test_cpuset
Docs/admin-guide/cgroup-v2: fix delay_nsec unit in io.latency doc
selftests/cgroup: Remove redundant cg_enter_current() call in test_core
selftests/cgroup: Add test for cpuset affinity on controller disable
cgroup/cpuset: Handle the special case of non-moving tasks in cpuset_can_attach()
cgroup/cpuset: Support multiple destination cpusets for cpuset_*attach()
selftests/cgroup: fix missing TAP output in test_hugetlb_memcg
cgroup/cpuset: Support multiple source cpusets for cpuset_*attach()
cgroup/cpuset: Move mpol_rebind_mm/cpuset_migrate_mm() calls inside cpuset_attach_task()
cgroup/cpuset: Make attach_ctx.old_cs track task group leader
...
|
|
Linux 7.2
There was a lot of conflicts this round between fixes and next,
and I'd like to get the merge resolutions that we have in drm-tip.
Signed-off-by: Dave Airlie <airlied@redhat.com>
|
|
Pull in dependents, the flat hierarchy fix depends on this.
Signed-off-by: Peter Zijlstra <peterz@infradead.org>
|
|
Commit 16a1d968358a ("mm/slab: remove mm/slab.c and slab_def.h")
removed the SLAB allocator, the only allocator that implemented cpuset
slab spreading. Commit 61a182ab61a6 ("cgroup/cpuset: Remove
cpuset_do_slab_mem_spread()") then removed the last task_spread_slab()
caller. Commit 3ab67a9ce82f ("cgroup/cpuset: Mark memory_spread_slab as
obsolete") marked the legacy control obsolete.
cpuset still updates PFA_SPREAD_SLAB when tasks attach to a legacy
cpuset and walks all tasks in a cpuset when memory_spread_slab changes.
Remove the unused task flag and its helpers, and make spread task
updates depend only on memory_spread_page.
Keep the memory_spread_slab control and CS_SPREAD_SLAB state so legacy
users retain the existing write, readback and inheritance behavior.
Update the comments and documentation to describe only page-cache
spreading as functional.
Assisted-by: LLM
Signed-off-by: Guopeng Zhang <zhangguopeng@kylinos.cn>
Reviewed-by: Waiman Long <longman@redhat.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
Pull bpf-next d114bb989367 ("Merge branch
'add-arena-argument-support-to-kfuncs-and-struct_ops'") to make the __arena
and __arena__nullable kfunc and struct_ops argument suffixes available. The
suffixed arguments will be used to convert sched_ext kfuncs and struct_ops
callbacks that currently pass arena pointers as scalars and rebase them by
hand.
|
|
cpuset_partition_show() reads cs->prs_err without cpuset_mutex using
READ_ONCE(). The field is documented as not lock protected, but several
updates to live cpusets still use plain stores.
Convert the remaining prs_err stores on live cpusets to WRITE_ONCE().
Fixes: 0c7f293efc87 ("cgroup/cpuset: Add cpuset.cpus.exclusive.effective for v2")
Assisted-by: LLM
Signed-off-by: Guopeng Zhang <zhangguopeng@kylinos.cn>
Reviewed-by: Waiman Long <longman@redhat.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
These comments describing the page allocator are out of date:
- __alloc_pages() is no longer a public API and has no business being
described outside of mm/.
- The `wait` variable is gone.
It may be out of date for other reasons too but this patch is just
fixing the issues that stood out.
To fix it:
- Instead of referring to a specific function, instead to "the page
allocator"
- Completely drop out-of-date details of that function's internal
behaviour, since they were irrelevant anyway.
Link: https://lore.kernel.org/20260715-spin-trylock-followup-v3-2-fc4d246f705d@google.com
Signed-off-by: Brendan Jackman <jackmanb@google.com>
Suggested-by: Zi Yan <ziy@nvidia.com>
Link: https://lore.kernel.org/all/DJP11T5V7BDW.2FZZZ8R6LOY4I@nvidia.com/
Reviewed-by: Zi Yan <ziy@nvidia.com>
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Acked-by: Tejun Heo <tj@kernel.org>
Cc: David Hildenbrand <david@kernel.org>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Michal Koutný <mkoutny@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Waiman Long <longman@redhat.com>
Cc: Brendan Jackman <brendan.jackman@linux.dev>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Add an optional reclaim callback to struct dmem_cgroup_region. When
dmem.max is set below the current usage of a cgroup pool, the new limit
is applied immediately (so that concurrent allocations are throttled
while reclaim is in progress) and then the driver is asked to evict
memory to bring usage back below the limit.
Reclaim is attempted up to a bounded number of times. No error is
returned to userspace if usage remains above the limit after reclaim,
and a pending signal will abort the reclaim loop early. This matches
the behavior of memory.max in the memory cgroup controller.
Also honor O_NONBLOCK so that if that flag is set during the
max value write, no reclaim is initiated. The idea is to avoid
charging the reclaim cost to the writer of the max value.
v2:
- Write max before reclaim is attempted (Maarten)
- Let signals abort the reclaim without error (Maarten)
- If a new max value is written with the O_NONBLOCK flag,
reclaim is not attempted (Maarten)
- Extract region from the pool parameter rather than
passing it explicitly to set_resource_xxx().
v3:
- Use an rw_semaphore (unregister_sem) to protect reclaim callbacks
against concurrent region unregistration: readers (reclaim) hold the
read side; dmem_cgroup_unregister_region() takes the write side to
drain in-flight callbacks before returning. (Sashiko-bot)
v5:
- Rebased on the introduction of struct dmem_cgroup_init.
- Use nonblock=true in reset_all_resource_limits() to avoid sleeping
inside rcu_read_lock() in dmemcs_offline(). (Sashiko-bot)
- Compare usage against the truncated limit value stored in cnt.max,
not the original u64. (Sashiko-bot)
- Use a DMEM_MAX_RECLAIM_RETRIES (16) retry budget instead of 5, matching
the memcg controller's MAX_RECLAIM_RETRIES. Only -ENOSPC (no progress)
counts against the retry budget; other errors terminate the loop
immediately.
v6:
- Fix dmem_cgroup_ops->reclaim docstring: -ENOSPC does not stop reclaim
immediately but is retried up to DMEM_MAX_RECLAIM_RETRIES times; only
other negative errors terminate the loop. (Sashiko-bot)
v7:
- Replace the per-region rw_semaphore with a static SRCU domain
(dmemcg_srcu). SRCU is a better fit than rwsem for this use: it
avoids the per-region lock overhead on every reclaim call, and
synchronize_srcu() at unregister time is a rare operation. (Maarten)
- Trim in-function comments to focus on what rather than how.
Assisted-by: GitHub_Copilot:claude-sonnet-4.6
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Reviewed-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Tested-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Link: https://patch.msgid.link/20260725100036.2372-4-thomas.hellstrom@linux.intel.com
Signed-off-by: Maarten Lankhorst <dev@lankhorst.se>
|
|
Replace the bare u64 size argument to dmem_cgroup_register_region() and
drmm_cgroup_register_region() with a const struct dmem_cgroup_init *
pointer. The struct currently carries only the size field, but using a
struct makes the API extensible: future callers can supply additional
initialization parameters without adding more positional arguments.
Update all in-tree callers (amdgpu, xe) to use a compound-literal
initializer.
v5:
- Commit introduced.
Assisted-by: GitHub_Copilot:claude-sonnet-4.6
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Reviewed-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Tested-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Link: https://patch.msgid.link/20260725100036.2372-3-thomas.hellstrom@linux.intel.com
Acked-by: Dave Airlie <airlied@redhat.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Maarten Lankhorst <dev@lankhorst.se>
|
|
This helps to find a common subtree of two resources, which is important
when determining whether it's helpful to evict one resource in favor of
another.
To facilitate this, add a common helper to find the ancestor of two
cgroups using each cgroup's ancestor array.
Tested-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Reviewed-by: Maarten Lankhorst <dev@lankhorst.se>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Natalie Vock <natalie.vock@gmx.de>
Link: https://patch.msgid.link/20260804-dmemcg-aggressive-protect-v8-2-07af96681bf8@gmx.de
|
|
Callers can use this feedback to be more aggressive in making space for
allocations of a cgroup if they know it is protected.
These are counterparts to memcg's mem_cgroup_below_{min,low}.
Reviewed-by: Maarten Lankhorst <dev@lankhorst.se>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Natalie Vock <natalie.vock@gmx.de>
Link: https://patch.msgid.link/20260804-dmemcg-aggressive-protect-v8-1-07af96681bf8@gmx.de
|
|
The trailing semicolon belongs at the point of use, not in the macro
definition. All uses have been verified to have their own semicolons.
This was found using the following Coccinelle semantic patch:
@r@
identifier i : script:ocaml() { String.lowercase_ascii i = i };
expression e;
@@
*#define i(...) e;
Signed-off-by: Julia Lawall <Julia.Lawall@inria.fr>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
The nodemasks in these structures may come from a variety of sources,
including tasks and cpusets - and should never be modified by any code
when being passed around inside another context.
Link: https://lore.kernel.org/20260609002919.3967782-1-gourry@gourry.net
Signed-off-by: Gregory Price <gourry@gourry.net>
Reviewed-by: Lorenzo Stoakes <ljs@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Barry Song <baohua@kernel.org>
Acked-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Tested-by: SeongJae Park <sj@kernel.org>
Acked-by: SeongJae Park <sj@kernel.org>
Acked-by: Waiman Long <longman@redhat.com>
Acked-by: Zi Yan <ziy@nvidia.com>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: Baoquan He <baoquan.he@linux.dev>
Cc: Brendan Jackman <jackmanb@google.com>
Cc: Chris Li <chrisl@kernel.org>
Cc: David Rientjes <rientjes@google.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Kairui Song <kasong@tencent.com>
Cc: Kemeng Shi <shikemeng@huaweicloud.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Michal Koutný <mkoutny@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Nhat Pham <nphamcs@gmail.com>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Wei Xu <weixugc@google.com>
Cc: Yuanchu Xie <yuanchu@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
RDMA device names are unique only within a network namespace, but an
RDMA cgroup can account resources for devices from multiple namespaces.
Duplicate names therefore make cgroup output ambiguous and can cause
limit writes to select the wrong device.
Use the system-wide RDMA device index to distinguish duplicate names
while preserving the existing UAPI for unique names. Reject ambiguous
name-only writes with -ENOTUNIQ and expose a complete device view to
administrators.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Link: https://patch.msgid.link/20260716132316.1495242-9-jiri@resnulli.us
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Accept only one "region value" pair entry for the dmem.max, dmem.min,
dmem.low files.
This changes the UAPI that otherwise accepted multiple lines for setting
multiple entries in one write. No existing user is known to rely on
writing multiple regions in a single write.
Processing multiple regions in dmemcg_limit_write() could quietly change
first limits before failing on a later one and returning an error to the
writer, with no indication some changes occurred.
Acked-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Eric Chanudet <echanude@redhat.com>
Reviewed-by: Maxime Ripard <mripard@kernel.org>
Reviewed-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Tested-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Link: https://patch.msgid.link/20260608-cgroup-dmem-write-single-region-v2-1-b0cd6c4ccf1b@redhat.com
Signed-off-by: Maarten Lankhorst <dev@lankhorst.se>
|
|
A subsystem can attach to the cgroup hierarchy itself, independent of which
controllers are enabled where - BPF hooks already behave this way and
sched_ext sub-schedulers do too. Controller callbacks can't track task
migrations for them: sched_ext must re-home a task whose migration crosses a
sub-scheduler boundary, but the cpu controller's attach callbacks fire only
when the task_group changes and miss moves whenever the controller topology
is coarser than the sub-scheduler topology.
Add cgroup_task_notifier with per-task migration events mirroring the
can_attach/attach/cancel_attach phases so that a consumer which prepares
per-task state can also veto a migration: CGROUP_TASK_MIGRATING fires
pre-commit, CGROUP_TASK_MIGRATED post-commit and
CGROUP_TASK_MIGRATE_CANCELED unwinds a failed migration. Only migrations
that change a task's dfl cgroup are reported.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
With cgroup v2 migration of a multithreaded process having threads
in different cgroups of a threaded subtree, it is possible that
cpuset_can_attach() can be called with tasks that are not migrating with
respect to cpuset if cpuset controller is not enabled in some of the
subtree cgroups. IOW, the old cpuset can be the same as the new one. This
can cause problem when we need to track the set of old cpusets and the
new cpusets in singly linked lists as a cpuset cannot be in both lists.
As reported by Tejun, the following is an example threaded subtree with
partial cpuset delegation that can cause this issue to show up.
P (+cpuset)
|- R (cpuset) <- destination
| `- C (no cpuset) -> effective cpuset == R
`- W (cpuset)
Group leader in R, thread_a in C, thread_b in W; migrate the whole
process into R (echo $PID > R/cgroup.procs). thread_a moves C->R:
its cgroup changes so compare_css_sets() keeps it in the taskset, but
its cpuset css is unchanged (C inherits R's), so task_cs() == cs ==
R. cpuset is in ss_mask because thread_b (W->R) changed. can_attach()
then tags R as a source (thread_a) and the destination (thread_b):
Handle this special case by skipping tasks that are not migrating in
cpuset_can_attach() and avoid calling cpuset_can_attach_check() in this
case. By doing so, the destination cpuset will not be put into source
cpuset linked list.
As the source cpuset cannot be easily determined in cpuset_attach(),
unnecessary work can be performed if a task is not actually
migrating. However, no harm will be done except wasting some
CPU cycles. If it happens that none of the tasks is migrating,
attach_ctx.old_cs will be NULL and task iteration won't be needed.
Reported-by: Tejun Heo <tj@kernel.org>
Closes: https://lore.kernel.org/lkml/e254af713b5345aec3d086771ecf1e71@kernel.org
Signed-off-by: Waiman Long <longman@redhat.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
The only case where the cgroup_taskset structure requires task migration
to multiple cpusets is when enabling a cpuset controller in cgroup v2
where the newly created child cpusets inherits the same effective CPUs
and memory nodes from the parent. In that case, task migration can happen
directly with no update to tasks' CPU and memory nodes assignment and no
further work needed from the cpuset side except updating nr_deadline_tasks
when DL tasks are involved and setting old_mems_allowed in the child
cpusets.
Do that by tracking all the destination cpusets with a new dst_cs_head
singly linked list. The reset_migrate_dl_data() function is integrated
into clear_attach_data() so that it can be used for both source and
destination cpusets.
A warning will be printed if there are multiple destination cpusets but
it is not on default hierarchy or when the CPUs or memory nodes change.
Signed-off-by: Waiman Long <longman@redhat.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
a5b98009f16d ("sched/psi: fix race between file release and pressure write")
made pressure_write() hold cgroup_mutex across psi_trigger_create(), which
forks the psimon kthread for the first rtpoll trigger. As kthread creation
depends on the whole fork path, the commit inadvertently created a lot of
unwanted locking dependencies from cgroup_mutex.
sched_ext got hit by one: its enable path blocks forks and then grabs
cgroup_mutex, so a pressure write racing a scheduler enable deadlocks, with
every other fork piling up behind.
Fix it by splitting trigger creation so that the worker is forked with
cgroup_mutex dropped and the kernfs active reference left broken. The latter
matters because rmdir and cgroup.pressure writes drain active references
under cgroup_mutex. Publishing the trigger last keeps error reporting
synchronous and preserves the of->priv lifetime rules.
The trigger registered in the first stage pins the group's rtpoll machinery
across the unlocked window, leaving only creation races to resolve. The
catch-up poll on installation covers scheduling attempts dropped while there
was no worker.
v2: Retagged sched/psi (was cgroup).
Fixes: a5b98009f16d ("sched/psi: fix race between file release and pressure write")
Cc: stable@vger.kernel.org
Cc: Edward Adam Davis <eadavis@qq.com>
Cc: Chen Ridong <chenridong@huaweicloud.com>
Reported-by: Matt Fleming <mfleming@cloudflare.com>
Closes: https://lore.kernel.org/all/20260710100441.2653477-1-matt@readmodwrite.com/
Signed-off-by: Tejun Heo <tj@kernel.org>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Tested-by: Matt Fleming <mfleming@cloudflare.com>
Acked-by: Suren Baghdasaryan <surenb@google.com>
|
|
There are 2 possible scenarios where the cgroup_taskset structure
passed into the cgroup can_attach() and attach() methods can contain
task migration data with multiple source cpusets.
- A multithread application with threads in different cpusets is
fully migrated into a new cpuset.
- Disabling v2 cpuset controller will move all the tasks in child
cpusets to the parent cpuset.
The current cpuset_can_attach() and cpuset_attach() functions still
expect task migration is from one source cpuset to one destination
cpuset.
Fix that by tracking the set of source (old) cpusets in singly linked
lists. The list will be iterated when necessary to properly update
internal data.
To ensure proper DL tasks accounting, the nr_migrate_dl_tasks in both
the source and destination cpusets are decremented/incremented with
their values added to nr_deadline_tasks when the migration is successful.
The setting of the global attach_ctx.cpus_updated and
attach_ctx.mems_updated flags are also moved from cpuset_attach()
to cpuset_can_attach() as the correct source cpuset can no longer be
determined in cpuset_attach() and cpuset states will not be changed
between cpuset_attach() and cpuset_can_attach() with an earlier patch.
Signed-off-by: Waiman Long <longman@redhat.com>
Reviewed-by: Ridong Chen <ridong.chen@linux.dev>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
The cpuset_attach_task() was introduced in commit 42a11bf5c543
("cgroup/cpuset: Make cpuset_fork() handle CLONE_INTO_CGROUP properly")
to enable the CLONE_INTO_CGROUP flag of clone(2) to behave more like
moving a task from one cpuset into another one. That commits didn't
move the mpol_rebind_mm() and cpuset_migrate_mm() calls for group leader
into cpuset_attach_task().
When the CLONE_INTO_CGROUP flag is used without CLONE_THREAD, the new
task is its own group leader. So it is still not equivalent to moving
task between cpusets in this case. Make CLONE_INTO_CGROUP behaves
more close to cpuset_attach() by moving the mpol_rebind_mm() and
cpuset_migrate_mm() calls inside cpuset_attach_task().
Also move the stack local cpus_updated, mems_updated and queue_task_work
flags into attach_ctx so that these flags can be accessed inside and
outside of cpuset_attach_task(). The cpuset_fork() function is updated
to set up these flags and do memory migration if necessary.
Reviewed-by: Ridong Chen <ridong.chen@linux.dev>
Signed-off-by: Waiman Long <longman@redhat.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
There are two possible ways that migration of tasks from multiple source
cpusets to a target cpuset can happen. Either a multithread application
with threads in different cpusets is wholely migrated to a new cpuset
or disabling of v2 cpuset controller will move all the tasks in child
cpusets to the parent cpuset.
In the former case, it is the mm setting of the group leader that
really matters. So attach_ctx.old_cs should track the oldcs of the
thread leader. In the latter case, effective_mems of child cpusets
must always be a subset of the parent. So no real page migration
will not be necessary no matter which child cpuset is selected as
attach_ctx.old_cs.
IOW, attach_ctx.old_cs should be updated to match the latest task
group leader in cpuset_can_attach(), but fall back to that of the first
task if there is no group leader in the taskset.
Suggested-by: Ridong Chen <ridong.chen@linux.dev>
Signed-off-by: Waiman Long <longman@redhat.com>
Reviewed-by: Ridong Chen <ridong.chen@linux.dev>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
Expand the scope of cpuset_can_attach_check() by including the setting
of setsched flag inside cpuset_can_attach_check() with the new @oldcs
and @psetsched argument. As cpuset_can_attach_check() is also called
from cpuset_can_fork(), set the new arguments to NULL from that caller.
Signed-off-by: Waiman Long <longman@redhat.com>
Reviewed-by: Ridong Chen <ridong.chen@linux.dev>
Signed-off-by: Michal Koutný <mkoutny@suse.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
Extract the DL bandwidth allocation code in cpuset_attach() to a new
cpuset_reserve_dl_bw() helper to simplify code.
No functional change is expected.
Signed-off-by: Waiman Long <longman@redhat.com>
Reviewed-by: Ridong Chen <ridong.chen@linux.dev>
Reviewed-by: Gregory Price <gourry@gourry.net>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
Put the task attach related cpuset_attach_old_cs and
cpuset_attach_nodemask_to static variables into the new attach_ctx
structure to improve readability and ease maintanence.
No functional change is expected.
Suggested-by: Ridong Chen <ridong.chen@linux.dev>
Signed-off-by: Waiman Long <longman@redhat.com>
Reviewed-by: Ridong Chen <ridong.chen@linux.dev>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
Commit e44193d39e8d ("cpuset: let hotplug propagation work wait for
task attaching") was introduced to let hotplug operation to wait
until the completion of task attach operation. However, it is still
possible that the states of the source or destination cpuset can
be changed between the cpuset_can_attach() call and the subsequent
cpuset_attach()/cpuset_cancel_attach() call.
As a result, data gathered during cpuset_can_attach() cannot be reliably
used in the subsequent cpuset_attach()/cpuset_cancel_attach()
call at all. Make the task attach operation more robust
and allow the sharing of data between cpuset_can_attach() and
cpuset_attach()/cpuset_cancel_attach() by making cpuset_write_resmask()
and cpuset_partition_write() wait for the completion of task attach
as well.
Ideally, an ongoing task attach operation should block any cpuset write
operation that can change its internal state until the operation is
completed. However, the attach_in_progress flag is currently per cpuset
and only the destination cpuset will have this flag set. The flag is not
set in the source cpuset where the tasks will be moved from. Even if we
extend the scope to include the source cpuset, it will not block cpuset
operation that changes the state of one of its ancestor cpuset which may
indirectly impact the state of the source or destination cpuset. It may
be too costly to set the flag for the whole subtree, it is far easier
to just make the flag global and block all the cpuset write operation
whenever a task attach operation is in progress.
Make that change by creating a new cpuset attach context (attach_ctx)
structure to hold the global in_progress flag and use it for blocking
cpuset write operation if a cpuset attach operation is in progress. Also
add a new wait_attach_done_lock() helper to do the waiting for an
ongoing attach operation and acquire the cpuset_mutex.
The comments about validate_change() are no longer valid as it won't
be called at all if an attach operation is in progress. So the comments
can be removed.
The per-cpuset attach_in_progress flag is also currently used in
partition_is_populated() and cpuset_is_populated() to determine if
an empty cpuset will have incoming task. This check will no longer be
needed as this function will not be called when there is a task attach
in progress. So the flag check is now removed.
Reviewed-by: Ridong Chen <ridong.chen@linux.dev>
Signed-off-by: Waiman Long <longman@redhat.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
Whenever memory node mask is changed, there are 4 places where the node
mask has to be updated or used.
1) task's node mask via cpuset_change_task_nodemask()
2) memory policy binding via mpol_rebind_mm()
3) if memory migration is enabled, migrate from old_mems_allowed to
the new node mask via cpuset_migrate_mm().
4) setting old_mems_allowed
These memory actions are done in cpuset_update_tasks_nodemask() and
cpuset_attach(). However there are inconsistencies in what node masks
are being used in these 2 functions.
In cpuset_update_tasks_nodemask(),
- cpuset_change_task_nodemask(): guarantee_online_mems()
- mpol_rebind_mm(): mems_allowed
- cpuset_migrate_mm(): guarantee_online_mems()
- old_mems_allowed: guarantee_online_mems()
In cpuset_attach(),
- cpuset_change_task_nodemask(): guarantee_online_mems()
- mpol_rebind_mm(): effective_mems
- cpuset_migrate_mm(): effective_mems
- old_mems_allowed: effective_mems
These inconsistencies dates back to quite a long time ago and it is
hard to say what should be the correct values.
The guarantee_online_mems() function returns a node mask from current or
an ancestor cpuset that is a subset of node_states[N_MEMORY]. Nodes in
node_states[N_MEMORY] are all online, i.e. in node_states[N_ONLINE].
However, node in node_states[N_ONLINE] may not have memory. So
node_states[N_MEMORY] should be a subset of node_states[N_ONLINE].
The guarantee_online_mems() function should mostly be useful for v1
where mems_allowed is the same as effective_mems. With v2, the memory
nodes in effective_mems should be a subset of node_states[N_MEMORY]
except when a memory hot-unplug operation is in progress and a memory
node is removed from node_states[N_MEMORY] but not yet reflected in
the effective_mems's as cpuset_handle_hotplug() has not been called
from cpuset_track_online_nodes().
Let use the following setup for both of them and make them consistent.
- cpuset_change_task_nodemask(): guarantee_online_mems()
- mpol_rebind_mm(): effective_mems
- cpuset_migrate_mm(): guarantee_online_mems()
- old_mems_allowed: guarantee_online_mems()
So for v2, it is effectively all effective_mems most of the time. For
v1, mpol_rebind_mm() uses mems_allowed which may differ from what
guarantee_online_mems() returns, but it conforms to what the cpuset v1
documentation says with respect to setting memory policy.
Signed-off-by: Waiman Long <longman@redhat.com>
Reviewed-by: Ridong Chen <ridong.chen@linux.dev>
Reviewed-by: Gregory Price <gourry@gourry.net>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
The nr_deadline_tasks variable in the cpuset structure was introduced by
commit 6c24849f5515 ("sched/cpuset: Keep track of SCHED_DEADLINE task
in cpusets"). It is reported by sashiko [1] that nr_deadline_tasks
can currently be modified by inc_dl_tasks_cs() under rq->lock and
by cpuset_attach() under cpuset_mutex. So if both updates happen
simultaneously, the nr_deadline_tasks variable can be corrupted leading
to incorrect operations down the road.
Fix that by changing its type to atomic_t so that nr_deadline_tasks
are always atomically updated. This fix patch is a low hanging fruit.
It can handle some of the races between a concurrent sched_setscheduler()
and cpuset_can_attach()/cpuset_attach() calls, but not all of them like
the other issue raised by sashiko [2]. This will be handled hopefully
in a future follow up patch.
[1] https://sashiko.dev/#/patchset/20260626181923.133658-1-longman%40redhat.com
[2] https://sashiko.dev/#/patchset/20260630033344.352702-1-longman%40redhat.com
Fixes: 6c24849f5515 ("sched/cpuset: Keep track of SCHED_DEADLINE task in cpusets")
Reviewed-by: Ridong Chen <ridong.chen@linux.dev>
Signed-off-by: Waiman Long <longman@redhat.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
Creating a child cpuset where cpuset.mems is never set leads to a div/0
when a VMA mempolicy with MPOL_F_RELATIVE_NODES rebinds in response to a
CPU hotplug event.
Reproduction steps:
1) Create a cgroup w/ cpuset controls (do not set cpuset.mems)
2) Move the task into the child cpuset
3) Create a VMA mempolicy for that task with MPOL_F_RELATIVE_NODES
4) unplug and hotplug a cpu
echo 0 > /sys/devices/system/cpu/cpu1/online
echo 1 > /sys/devices/system/cpu/cpu1/online
5) mempolicy rebind does a div/0 in mpol_relative_nodemask on the
call to __nodes_fold()
The cpuset code passes (cs->mems_allowed) which is not guaranteed to have
nodes to the rebind routine. Use cs->effective_mems instead, which is
guaranteed to have a non-empty nodemask once we reach that code path.
Link: https://lore.kernel.org/all/CA+0ovCiEz6SP_sn3kN4Tb+_oC=eHMXy_Ffj=usV3wREdQrUtww@mail.gmail.com/
Fixes: ae1c802382f7 ("cpuset: apply cs->effective_{cpus,mems}")
Closes: https://lore.kernel.org/linux-mm/CA+0ovCgxbZkXa+OU8w3s84R3KNPNxxRfmsNR-udh+afQBbGNmw@mail.gmail.com/
Suggested-by: Gregory Price <gourry@gourry.net>
Suggested-by: Waiman Long <longman@redhat.com>
Acked-by: Waiman Long <longman@redhat.com>
Signed-off-by: Farhad Alemi <farhad.alemi@berkeley.edu>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Alistair Popple <apopple@nvidia.com>
Cc: Byungchul Park <byungchul@sk.com>
Cc: Gregory Price <gourry@gourry.net>
Cc: "Huang, Ying" <ying.huang@linux.alibaba.com>
Cc: Joshua Hahn <joshua.hahnjy@gmail.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Rakie Kim <rakie.kim@sk.com>
Cc: Rasmus Villemoes <linux@rasmusvillemoes.dk>
Cc: Zi Yan <ziy@nvidia.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Ridong Chen <ridong.chen@linux.dev>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: "Michal Koutný" <mkoutny@suse.com>
Cc: <stable@vger.kernel.org>
[ david: add a comment, slightly rephrase description ]
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
In order to avoid the average CPU fraction avg(F_g_n) becoming tiny '1/N',
assume each cgroup is maximally concurrent and distrubute 'N*weight', such
that:
F_g_n' = N * F_g_n
Giving:
avg(F_g_n') = N*avg(F_g_n) ~ N * 1/N = 1
And while this sounds like it solves things, remember what that ~ meant. There
is the corner case when a cgroup is minimally loaded, eg a single runnable
task, therefore limit the CPU fraction to that of a nice -20 task to avoid
getting too much load.
This last bit is what makes it different from a previous proposal to allow
raising cpu.weight to '100 * N', that would not limit the mininal concurrency
case and results in a very large F_g_n. And just like F_g_n << 1 is
problematic, so is F_g_n >> 1 for the exact same reasons (it would drown the
kthreads, but it also risks overflowing the load values).
So while this might appear to be a better scheme than the current default
scheme, it doesn't really handle less than maximal concurrency nicely -- it
clips and introduces artificially large weights. So where the traditional SMP
mode works well when nr_tasks << nr_cpus, MAX doesn't work well in that regime
and vice-versa.
The meaning of "cpu.weight" would be: weight per allowed CPU.
Included for completeness (and infrastructure).
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260605124051.589618504%40infradead.org
|
|
Backmerging to get drm-misc-next to v7.2-rc1.
Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
|
|
As reported by sashiko [1], cpuset_update_tasks_nodemask() will do
mpol_rebind_mm() and possibly cpuset_migrate_mm() for all threads of
a multithreaded process. Since commit 3df9ca0a2b8b ("cpuset: migrate
memory only for threadgroup leaders"), cpuset_attach() had been updated
to rebind and migrate memory only for threadgroup leaders to mark the
group leader as the owner of the mm_struct.
To be consistent and avoid unnecessary performance overhead for heavily
multithreaded processes, follow the cpuset_attach() example and perform
memory rebind and migration only for threadgroup leaders.
Also add a paragraph in cgroup-v2.rst under cpuset.mems that the
threadgroup leader is the memory owner of that threadgroup. Therefore
the non-leading threads shouldn't be in other cgroups whose "cpuset.mems"
doesn't fully overlap that of the group leader.
[1] https://sashiko.dev/#/patchset/20260621032816.1806773-1-longman%40redhat.com
Signed-off-by: Waiman Long <longman@redhat.com>
Reviewed-by: Ridong Chen <ridong.chen@linux.dev>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
As reported by sashiko [1], cpuset_hotplug_update_tasks() may perform
unnecessary task iteration and updating of tasks' CPU and node masks
when mems_allowed and/or cpus_allowed are not set in cpuset v2. It is
due to the fact that the temporary new_cpus and new_mems masks do not
inherit parent's effective_cpus/mems when they are empty which is the
expected behavior for cpuset v2 since commit 4ec22e9c5a90 ("cpuset:
Enable cpuset controller in default hierarchy").
Fix that and avoid unnecessary work by enhancing
compute_effective_cpumask() to add the empty cpumask check
and inheriting the parent's versions if empty when in v2. A new
compute_effective_nodemask() helper is also added to perform a similar
function for new effective_mems.
Add new test_cpuset_prs.sh test cases to confirm that effective_cpus
will inherit the parent's version if cpuset.cpus is empty.
[1] https://sashiko.dev/#/patchset/20260621032816.1806773-1-longman%40redhat.com
Suggested-by: Ridong Chen <ridong.chen@linux.dev>
Fixes: 4ec22e9c5a90 ("cpuset: Enable cpuset controller in default hierarchy")
Signed-off-by: Waiman Long <longman@redhat.com>
Reviewed-by: Ridong Chen <ridong.chen@linux.dev>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
Pull cgroup updates from Tejun Heo:
- Last cycle deferred css teardown on cgroup removal until the cgroup
depopulated, so a css is not taken offline while tasks can still
reference it. Disabling a controller through cgroup.subtree_control
still had the same problem. This reworks the deferral from per-cgroup
to per-css so that path is covered too.
- New RDMA controller monitoring files: rdma.peak for per-device peak
usage and rdma.events / rdma.events.local for resource-limit
exhaustion. The max-limit parser was rewritten, fixing two input
parsing bugs.
- cpuset: fix a sched-domain leak on the domain-rebuild failure path
and skip a redundant hardwall ancestor scan on v2.
- Misc: pair the remaining lockless cgroup.max.* reads with WRITE_ONCE,
assorted selftest robustness fixes, and doc path corrections.
* tag 'cgroup-for-7.2' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup: (22 commits)
cgroup: Migrate tasks to the root css when a controller is rebound
docs: cgroup: Fix stale source file paths
cgroup/cpuset: Free sched domains on rebuild guard failure
cgroup: pair max limit READ_ONCE() with WRITE_ONCE()
selftests/cgroup: enable memory controller in hugetlb memcg test
cgroup/rdma: Drop unnecessary READ_ONCE() on event counters
cgroup: Defer kill_css_finish() in cgroup_apply_control_disable()
cgroup: Add per-subsys-css kill_css_finish deferral
cgroup: Move populated counters to cgroup_subsys_state
cgroup: Annotate unlocked nr_populated_* accesses with READ_ONCE/WRITE_ONCE
cgroup: Inline cgroup_has_tasks() in cgroup.h
cgroup/rdma: document rdma.peak, rdma.events and rdma.events.local
cgroup/rdma: add rdma.events.local for per-cgroup allocation failure attribution
cgroup/rdma: add rdma.events to track resource limit exhaustion
cgroup/rdma: add rdma.peak for per-device peak usage tracking
selftests/cgroup: check malloc return value in alloc_anon functions
cgroup/cpuset: Skip hardwall ancestor scan in cpuset v2 in cpuset_current_node_allowed()
selftests/cgroup: fix misleading debug message in test_cgfreezer_time_child
selftests/cgroup: fix child process escaping to parent cleanup in test_cpucg_nice
selftests/cgroup: Add NULL check after malloc in cgroup_util.c
...
|
|
Just like we have memory.peak, introduce a dmem.peak, which uses the
page_counter support for that.
For now, make it read-only.
This allows for memory usage monitoring without polling dmem.current when
the information needed is the maximum device memory used. That can be used
for capacity planning, such that dmem.max can be properly setup for a given
workload. It can also be used for debugging to determine whether a given
workload would have caused eviction or system memory use.
Signed-off-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Reviewed-by: Michal Koutný <mkoutny@suse.com>
Reviewed-by: Maarten Lankhorst <dev@lankhorst.se>
Link: https://patch.msgid.link/20260514-dmem_peak-v3-1-b64ce5d3ac38@igalia.com
Signed-off-by: Maarten Lankhorst <dev@lankhorst.se>
|
|
Pull cgroup fixes from Tejun Heo:
"One cpuset fix and a maintenance update, both low-risk:
- Fix cpuset partition CPU accounting under sibling CPU exclusion
that could produce wrong CPU assignments and trigger
scheduling-domain warnings. Includes selftests.
- Update an email address in MAINTAINERS"
* tag 'cgroup-for-7.1-rc6-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup:
cgroup/cpuset: Change Ridong's email
cgroup/cpuset: Add test cases for sibling CPU exclusion on partition update
cgroup/cpuset: Use effective_xcpus in partcmd_update add/del mask calculation
|
|
cgroup_apply_control_disable() defers kill_css_finish() while a css is
still populated, relying on css_update_populated() to fire the deferred
kill once the populated count reaches zero.
This deadlocks when a controller is rebound out of a hierarchy. Mounting
an implicit_on_dfl controller such as perf_event as a v1 hierarchy steals
it off the default hierarchy, and rebind_subsystems() kills its
per-cgroup csses while they are still populated. The migration run in the
same step keeps the old css for a controller no longer in the hierarchy's
mask, so no task is migrated off the dying csses. Their populated count
never reaches zero, the deferred kill_css_finish() never fires, and the
next cgroup_lock_and_drain_offline() hangs forever under cgroup_mutex.
That migration is already a no-op pass over the rebound subtree. Add
cgroup_rebind_ss_mask so find_existing_css_set() resolves the leaving
controllers to the root css. Their tasks are migrated there, the
per-cgroup csses depopulate, and cgroup_apply_control_disable() kills
them synchronously. The deferral stays correct for the rmdir and
controller-disable paths it was meant for.
Fixes: 1dffd95575eb ("cgroup: Defer kill_css_finish() in cgroup_apply_control_disable()")
Reported-by: Mark Brown <broonie@kernel.org>
Closes: https://lore.kernel.org/all/41cd159c-54e5-45e0-81df-eaf36a6c028e@sirena.org.uk/
Reported-by: Bert Karwatzki <spasswolf@web.de>
Closes: https://lore.kernel.org/all/4e986b4ed7e16547805d54b6e67d09120bc4d2f2.camel@web.de/
Tested-by: Mark Brown <broonie@kernel.org>
Tested-by: Bert Karwatzki <spasswolf@web.de>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
generate_sched_domains() returns sched-domain masks and optional
attributes that are normally handed to partition_sched_domains(), which
takes ownership of them.
rebuild_sched_domains_locked() has a WARN guard after
generate_sched_domains() and before partition_sched_domains() to avoid
passing offline CPUs into the scheduler domain rebuild path. If that
guard fires, the function currently returns directly without freeing
the generated doms and attr.
Free the generated sched-domain masks and attributes before returning
from the guard failure path.
Signed-off-by: Guopeng Zhang <zhangguopeng@kylinos.cn>
Reviewed-by: Waiman Long <longman@redhat.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
cgroup.max.descendants and cgroup.max.depth are shown through seq_file.
Their show callbacks read cgrp->max_descendants and cgrp->max_depth with
READ_ONCE(), respectively.
The corresponding write callbacks update the same scalar fields while
holding the cgroup lock, but the seq_file show path does not serialize
against those stores. This leaves the lockless show-side loads annotated
with READ_ONCE(), while the corresponding stores remain plain stores.
Use WRITE_ONCE() for the updates so the intended lockless access is marked
consistently on both sides. This does not change locking, ordering, or
user-visible semantics.
Assisted-by: OpenAI-Codex:gpt-5.5
Signed-off-by: Ren Tamura <ren.tamura.oss@gmail.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
When sibling CPU exclusion occurs, a partition's user_xcpus may contain
CPUs that were never actually granted to it. These CPUs are present in
user_xcpus(cs) but not in cs->effective_xcpus.
The partcmd_update path in update_parent_effective_cpumask() uses
user_xcpus(cs) (via the local variable xcpus) to compute the addmask
(CPUs to return to parent) and delmask (CPUs to request from parent).
This is incorrect:
1) When newmask removes a CPU that was previously excluded by a
sibling, addmask incorrectly includes that CPU and tries to return
it to the parent even though the partition never actually owned it,
causing CPU overlap with sibling partitions and triggering warnings
in generate_sched_domains().
2) When newmask adds a previously excluded CPU that is now available,
delmask fails to request it from the parent because user_xcpus(cs)
already includes it.
Fix this by using cs->effective_xcpus instead of user_xcpus(cs) in all
partcmd_update paths that calculate addmask or delmask, including the
PERR_NOCPUS error handling paths.
Reproducers:
Example 1 - Removing a sibling-excluded CPU incorrectly returns it:
# cd /sys/fs/cgroup
# echo "0-1" > a1/cpuset.cpus
# echo "root" > a1/cpuset.cpus.partition
# echo "0-2" > b1/cpuset.cpus
# echo "root" > b1/cpuset.cpus.partition
# echo "2" > b1/cpuset.cpus
# cat cpuset.cpus.effective
# Actual: 0-1,3 Expected: 3
Example 2 - Expanding to a previously excluded CPU fails to request it:
# cd /sys/fs/cgroup
# echo "0-1" > a1/cpuset.cpus
# echo "root" > a1/cpuset.cpus.partition
# echo "0-2" > b1/cpuset.cpus
# echo "root" > b1/cpuset.cpus.partition
# echo "member" > a1/cpuset.cpus.partition
# echo "1-2" > b1/cpuset.cpus
# cat cpuset.cpus.effective
# Actual: 0-1,3 Expected: 0,3
Fixes: 2a3602030d80 ("cgroup/cpuset: Don't invalidate sibling partitions on cpuset.cpus conflict")
Cc: stable@vger.kernel.org # v7.0+
Suggested-by: Zhang Guopeng <zhangguopeng@kylinos.cn>
Signed-off-by: Sun Shaojie <sunshaojie@kylinos.cn>
Reviewed-by: Waiman Long <longman@redhat.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
Pull cgroup fixes from Tejun Heo:
"Two rstat fixes:
- Out-of-bounds access in the css_rstat_updated() BPF kfunc when
called with an unchecked user-supplied cpu
- Over-strict NMI guard after the recent switch to try_cmpxchg left
sparc and ppc64 unable to queue rstat updates from NMI"
* tag 'cgroup-for-7.1-rc4-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup:
cgroup: rstat: relax NMI guard after switch to try_cmpxchg
cgroup/rstat: validate cpu before css_rstat_cpu() access
|
|
Commit 36df6e3dbd7e ("cgroup: make css_rstat_updated nmi safe") used
this_cpu_cmpxchg() for the lockless insertion, and therefore required
both ARCH_HAVE_NMI_SAFE_CMPXCHG and ARCH_HAS_NMI_SAFE_THIS_CPU_OPS in
the NMI guard: on archs without the latter, this_cpu_cmpxchg() falls
back to "local_irq_save() + plain cmpxchg", and local_irq_save()
cannot mask NMIs.
Commit 3309b63a2281 ("cgroup: rstat: use LOCK CMPXCHG in
css_rstat_updated") later replaced this_cpu_cmpxchg() with plain
try_cmpxchg() to fix cross-CPU lockless-list corruption, but left the
NMI guard untouched. After that switch, css_rstat_updated() no longer
performs any this_cpu_*() RMW operations and only relies on the arch
having NMI-safe cmpxchg, so ARCH_HAS_NMI_SAFE_THIS_CPU_OPS is no
longer required in the guard.
Relax the guard accordingly so that archs which have HAVE_NMI and
ARCH_HAVE_NMI_SAFE_CMPXCHG but not ARCH_HAS_NMI_SAFE_THIS_CPU_OPS
(e.g. sparc, powerpc on PPC64/BOOK3S) can benefit from the existing
CONFIG_MEMCG_NMI_SAFETY_REQUIRES_ATOMIC path. Without this, the css
is never queued in NMI on those archs, and the atomics staged by
account_{slab,kmem}_nmi_safe() are not drained by flush_nmi_stats().
Fixes: 3309b63a2281 ("cgroup: rstat: use LOCK CMPXCHG in css_rstat_updated")
Signed-off-by: Cunlong Li <shenxiaogll@gmail.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
css_rstat_updated() is exposed as a BPF kfunc and accepts a
caller-provided cpu argument. The function uses cpu for per-cpu rstat
lookups without checking whether it refers to a valid possible CPU.
A BPF iter/cgroup program with CAP_BPF and CAP_PERFMON can pass an
invalid cpu value. On an unfixed UBSCAN_BOUNDS test kernel, cpu ==
0x7fffffff triggers:
UBSAN: array-index-out-of-bounds in kernel/cgroup/rstat.c:31:9
index 2147483647 is out of range for type 'long unsigned int [64]'
Call Trace:
css_rstat_updated
bpf_iter_run_prog
cgroup_iter_seq_show
bpf_seq_read
Add cpu validation to the BPF-facing css_rstat_updated() kfunc and
move the common implementation to __css_rstat_updated() for in-kernel
callers.
Fixes: a319185be9f5 ("cgroup: bpf: enable bpf programs to integrate with rstat")
Signed-off-by: Qing Ming <a0yami@mailbox.org>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
All accesses to the event counters are serialized by rdmacg_mutex,
making the READ_ONCE() annotations unnecessary. Remove them.
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
Same race shape as the rmdir path that 93618edf7538 ("cgroup: Defer css
percpu_ref kill on rmdir until cgroup is depopulated") fixed: a task past
exit_signals() whose cset subsys[ssid] still pins the disabled controller's
css can be touching subsys state while ->css_offline() runs. The earlier
patches in this series built up the per-subsys-css deferral machinery and
routed cgroup_destroy_locked() through it. Apply the same shape to
cgroup_apply_control_disable():
kill_css_sync(css);
if (!css_is_populated(css))
kill_css_finish(css);
When the dying css is still populated, kill_css_finish() is deferred. The
walker in css_update_populated() fires kill_finish_work once the css's
hierarchical populated count drops to zero.
cgroup_lock_and_drain_offline()'s wait predicate switches from
percpu_ref_is_dying() to css_is_dying(). CSS_DYING is set by kill_css_sync()
and is a strict superset of percpu_ref_is_dying. Without this change, a +cpu
re-enable after a deferred -cpu disable would skip the drain (percpu_ref
isn't killed yet) and observe the still-CSS_DYING css through cgroup_css(),
treating it as live.
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
93618edf7538 ("cgroup: Defer css percpu_ref kill on rmdir until cgroup is
depopulated") deferred kill_css_finish() at the cgroup level: rmdir waits
for the entire cgroup's populated count to drop to zero, then fires
kill_css_finish() on every subsystem css at once. Replace that with
per-subsys-css deferral. Each subsystem css now tracks its own hierarchical
populated count and independently defers its kill_css_finish() until its own
subtree drains.
The rmdir-race fix carries through unchanged in shape. The dying css's
->css_offline() still waits until no PF_EXITING task references it, and v2's
cgroup-level machinery goes away.
cgroup_apply_control_disable() has the same race shape (PF_EXITING tasks
pinning a css whose ->css_offline() is about to run) and stays synchronous
here. This patch lays the groundwork for fixing it - per-cgroup waiting
can't gate one subsys css being killed while the rest of the cgroup stays
live, but per-css can.
Subtree-wide invariant preserved: a dying ancestor css stays populated
through nr_populated_children until every dying descendant's task drains, so
the walker fires the ancestor's kill_finish_work only after all descendants
have drained.
Add paired smp_mb()s in kill_css_sync() and css_update_populated() to fence
the StoreLoad on (CSS_DYING, populated counter), guaranteeing that either
the walker queues kill_finish_work or the caller fires synchronously.
cgroup_destroy_locked() was implicitly fenced by an unrelated css_set_lock
pair; cgroup_apply_control_disable() in the next patch is not.
Signed-off-by: Tejun Heo <tj@kernel.org>
|
Linus Torvalds
Dave Airlie
Peter Zijlstra
Guopeng Zhang
Tejun Heo
Brendan Jackman
Thomas Hellström
Natalie Vock
Julia Lawall
Gregory Price
Jiri Pirko
Eric Chanudet
Waiman Long
Farhad Alemi
Thomas Zimmermann
Thadeu Lima de Souza Cascardo
Ren Tamura
Sun Shaojie
Cunlong Li
Qing Ming
Tao Cui