| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Pull btrfs fixes from David Sterba:
"Among the regular fixes, there are two that were reported recently and
have user impact:
- filesystem id is now stable again on the default and common case
(it broke openconnect key derivation, while this is not secure,
it's still in use), the intention was to change id for the
temp_fsid use case
- fix detection of /dev/root and rename it after device scan, this
broke booting of initramdisk-less system with grub2 as the probe
needs the real device
Regular fixes:
- don't store compressed inline extent if the size is larger than
uncompressed
- in zoned mode, handle activation of zones for all supported block
group profiles in case there are still free ones left
- check space for a chunk item when reading sys array from superblock
- allow using space reserves when removing verity items fails
- fix error handling after free space tree rebuild fails
- abort transaction if reflink or hole punching fails and it's not
possible to update the inode
- properly protect block group iteration during device replace start
- error message fixups"
* tag 'for-7.3-rc3-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux:
btrfs: derive f_fsid with dev_t only when temp_fsid is active
btrfs: add "/dev/root" exception for device path update
btrfs: check if there is space for chunk item when validating sys chunk array
btrfs: abort transaction on failure to update inode for hole punching and reflinking
btrfs: clear free space tree creation state on rebuild failure
btrfs: handle lack of space when cleaning up verity items
btrfs: fix creation of compressed inline extents that don't save space
btrfs: tree-checker: fix error message regarding free space extent items
btrfs: tree-checker: print dev extent offset in error message
btrfs: take commit root semaphore when iterating in mark_block_group_to_copy()
btrfs: zoned: handle RAID profiles in btrfs_can_activate_zone()
|
|
Commit c2a74ed0494c ("btrfs: derive f_fsid from on-disk fsid and dev_t")
mixed dev_t into f_fsid for all single-device setups to avoid f_fsid
collisions with cloned filesystems.
However, doing this unconditionally breaks backward compatibility.
statfs(2) f_fsid changes after a kernel upgrade, and also can shift
across reboots or dev re-attaches as dev_t values change.
Fix this by only mixing dev_t when temp_fsid is active. This means for
non-temp_fsid setups or the original mount, we use the old method of
deriving fsid based on the UUID.
So in the case of a cloned Btrfs filesystem, we won't be able to
maintain the same fsid across mount recycle if the mount order changes.
Reported-by: Dave Hansen <dave.hansen@intel.com>
Link: https://lore.kernel.org/linux-btrfs/be0c08f5-2f31-40f5-8a3b-f2f58b3e00ff@intel.com
Fixes: c2a74ed0494c ("btrfs: derive f_fsid from on-disk fsid and dev_t")
CC: stable@vger.kernel.org # 7.2
Signed-off-by: Anand Jain <asj@kernel.org>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
[BEHAVIOR CHANGE]
Since commit 108cc8733989 ("btrfs: fix a lockdep caused by path
resolution during device scan"), users with btrfs rootfs but without an
initramfs are complaining that grub2 can no longer detect the rootfs
device:
/usr/sbin/grub-probe: error: cannot find a device for / (is /dev mounted?).
[CAUSE]
Although using btrfs without an initramfs is not recommended (if a new
device is added to the rootfs, the system can no longer boot, as there
is no way to register all devices), there is still a minority of users
doing this.
If there is no initramfs but the rootfs is on a block-device-based
filesystem, the kernel boot sequence initializes a minimal ramfs/tmpfs,
creates "/dev/root" with the proper device number for the rootfs, and
then invokes mount using "/dev/root".
That's why the end user will get the mount output:
/dev/root on / rw
To be honest, this is a user space problem: no one should trust the
device path shown in mount, only the device number.
E.g. one can even use "/proc/self/fd/*" to mount an fs, and that proc
path will be registered, and no one else can mount that fs using that
path.
Before commit 108cc8733989 ("btrfs: fix a lockdep caused by path
resolution during device scan"), btrfs had an internal path lookup
workaround to address such weird paths, it works by checking if the
existing device path can still resolve to the device number.
But that path resolution is deadlock prone, thus it's replaced by a
simple devt check.
This works fine in most cases, as a btrfs device is registered by udev at
boot time, thus all paths are sane.
However this will not work for systems without an initramfs, causing the
unreachable "/dev/root" path to exist forever without a way to rename it.
[WORKAROUND]
Add an exception to the device path rename requirement.
If the device has the name "/dev/root", we know it's booted without an
initramfs, and only for that case we allow device path update.
And if someone intentionally created "/dev/root" after boot, the
existing devt checks will reject that weird name as usual.
This should satisfy the minority of users, and still keep most of the
existing guards preventing unexpected/unnecessary device path updates.
Fixes: 108cc8733989 ("btrfs: fix a lockdep caused by path resolution during device scan")
Link: https://lore.kernel.org/linux-btrfs/CAKLYgeL7nrA4nXcewdv9Fqg_s=3GS=vmoypnEiZBKQ7rySZFuQ@mail.gmail.com/
Link: https://lore.kernel.org/linux-btrfs/dfbe1e27-dab8-4d55-8cf3-0b28eeac5df4@gmail.com/
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
We checked if have enough remaining space for a key before dereferencing a
key, but we then dereference a chunk item, to get the number of stripes,
without checking if there is space for the item. So add a check to see if
there is enough space for a chunk item before dereferencing the item to
extract the stripe count.
Fixes: 2a9bb78cfd36 ("btrfs: validate system chunk array at btrfs_validate_super()")
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
If we fail to update the inode we error out without aborting the
transaction, which can result in a persistent inconsistency if after
the failure the transaction is committed, as we have dropped file
extent items from a range and either punched a hole or insert a new file
extent item for that range (for reflinks).
So add the missing transaction abort.
Fixes: 2aaa66558172 ("Btrfs: add hole punching")
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
btrfs_rebuild_free_space_tree() sets BTRFS_FS_CREATING_FREE_SPACE_TREE
before rebuilding the free space tree. Several error paths return
without clearing this flag.
The transaction restart failure path can leave the flag set on a live
filesystem, causing delayed reference processing to be skipped. Clear it
on all free space tree rebuild failure paths. Keep
BTRFS_FS_FREE_SPACE_TREE_UNTRUSTED set, since a failed rebuild leaves
the free space tree untrusted. Callers must fall back to extent-tree
caching.
Fixes: 882af9f13e83 ("btrfs: handle free space tree rebuild in multiple transactions")
CC: stable@vger.kernel.org # 6.14+
Assisted-by: LLM
Reviewed-by: Boris Burkov <boris@bur.io>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Guanghui Yang <3497809730@qq.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
When enable_verity() hits the qgroup limit, rollback_verity() needs its
own metadata reservation. When the qgroup limit or lack of space refuses
the rollback, the whole filesystem is forced read-only even though the
qgroup limit was for one subvolume only. Also orphan cleanup at the next
mount fails the same way, so the leftover items are never removed: with
-EDQUOT the subvolume stays unreachable, and with -ENOSPC on a full
filesystem the next read-write mount fails.
Start transactions with btrfs_start_transaction_fallback_global_rsv() in
btrfs_orphan_cleanup(), drop_verity_items() and rollback_verity(). Those
calls only delete items and free the space in the end, so they may use
the global reserve and skip the qgroup limit, which avoids -ENOSPC and
-EDQUOT.
Fixes: 146054090b08 ("btrfs: initial fsverity support")
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Daniel Linjama <daniel@dev.linjama.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
If the compressed data of an inline extent is larger than or equals to the
size of the uncompressed data, we are still allowing the creation of the
compressed inline extent, which does not result in any benefits, quite the
contrary as we waste metadata space and have to decompress when reading.
This is a recent regression introduced in commit 3eaf5f082c4c ("btrfs:
extract inlined creation into a dedicated delalloc helper").
It happens because we are passing the block size to btrfs_compress_bio(),
so we don't get -E2BIG from the compression code anymore, but we can not
pass i_size either, because if i_size is smaller than sector size, we
end up never creating lzo compressed inline extent for such small i_size
values. So refuse the compressed result at run_delalloc_inline() if
its size is not smaller than the uncompressed size (i_size).
Reported-by: Hanabishi <i.r.e.c.c.a.k.u.n+kernel.org@gmail.com>
Link: https://lore.kernel.org/linux-btrfs/c97652a5-ac6b-4de6-aa23-3cdebc01d00b@gmail.com/
Fixes: 3eaf5f082c4c ("btrfs: extract inlined creation into a dedicated delalloc helper")
CC: stable@vger.kernel.org # 7.1+
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
The error message mentions a free space info item, but we are processing a
free space extent item, so fix the message.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
If a dev extent's offset is not sector size aligned, the error message is
printing the dev extent's objectid instead of the offset. This is a copy
paste error, as before this check we check the objectid field.
Fixes: 008e2512dc56 ("btrfs: tree-checker: add dev extent item checks")
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
mark_block_group_to_copy() iterates over the commit root with
skip_locking=true. A concurrent transaction commit can swap and free
the commit root during iteration, causing use-after-free when
accessing extent buffers.
Fix it by using path->need_commit_sem to protect the commit root search.
Fixes: 78ce9fc269af ("btrfs: zoned: mark block groups to copy for device-replace")
CC: stable@vger.kernel.org
Assisted-by: Codex:gpt-5.5
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: Hongling Zeng <zenghongling@kylinos.cn>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
btrfs_can_activate_zone() only accounts for the single and DUP profiles.
For a RAID0, RAID1, RAID1C3, RAID1C4 or RAID10 block group the profile
switch matches no case, so 'ret' stays false and the function reports
that no zone can be activated, even when the devices have plenty of
active zones left.
As a side effect BTRFS_FS_NEED_ZONE_FINISH gets set and, since
btrfs_can_activate_zone() bails out early once that bit is set, data
allocations will fail permanently: writers loop on -EAGAIN and hang in
btrfs_new_extent_direct() waiting for the bit to clear.
Each of these profiles needs one active zone per device, just like
single, so handle them the same way.
Reviewed-by: Boris Burkov <boris@bur.io>
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Pull btrfs fixes from David Sterba:
- tree-checker updates, validate values in b-tree item keys, other item
length checks
- don't do unnecessary transaction commit fallback when logging parent
directories
- in zoned mode, initialize space info of a block group early enough so
it does not lead to NULL pointer dereference
* tag 'for-7.3-rc2-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux:
btrfs: tree-checker: validate name length for extref items
btrfs: tree-checker: validate parent field for inode extref items
btrfs: tree-checker: validate key offset for inode ref keys
btrfs: fix unnecessary transaction commit fallback from btrfs_log_all_parents()
btrfs: set space_info before adding new free space in btrfs_make_block_group()
|
|
We are validating the name length of inode ref items, but we miss the same
validation for extref items. Sashiko pointed this out while reviewing
other patch. Add the missing validation, similar to what was done in commit
3dc22abc21f5 ("btrfs: tree-checker: validate INODE_REF's namelen").
Reviewed-by: Boris Burkov <boris@bur.io>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
For a subvolume tree, the parent field of an inode extref item corresponds
to an inode number, and that must always be within the range:
[ BTRFS_FIRST_FREE_OBJECTID (256), BTRFS_LAST_FREE_OBJECTID (-256) ]
Add a check for that in check_inode_extref().
Reviewed-by: Boris Burkov <boris@bur.io>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
For a subvolume tree, the offset of an inode ref key corresponds to an
inode number, and that must always be within the range:
[ BTRFS_FIRST_FREE_OBJECTID (256), BTRFS_LAST_FREE_OBJECTID (-256) ]
Add a check for that in check_inode_ref(). Sashiko complained about such
check missing in another unrelated patch.
Reviewed-by: Boris Burkov <boris@bur.io>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
When btrfs_log_all_parents() returns without doing any work (because all
parent directories were already logged), it returns 1, which is propagated
up the fsync call chain up to btrfs_log_dentry_safe(), and that causes
btrfs_sync_file() to trigger am unnecessary transaction commit.
This all happens because the call to btrfs_search_slot() in
btrfs_log_all_parents() always returns 1, as there can not be any inode
ref keys with an offset 0 (an invalid inode number), so if the while loop
below it does not do any work because all parent directories were already
logged, the 'ret' variable remains with a value of 1, which is then
returned up the call chain to btrfs_sync_file().
Fix this by setting 'ret' to 0 after the call to btrfs_search_slot().
Fixes: 0f24ea456ae1 ("btrfs: tracepoints: add trace event for btrfs_log_all_parents()")
Reviewed-by: Boris Burkov <boris@bur.io>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
btrfs_make_block_group() calls btrfs_add_new_free_space() before
assigning cache->space_info. On a zoned filesystem that ends up in
__btrfs_add_free_space_zoned(), which dereferences
block_group->space_info and thus hits a NULL pointer dereference when a
non-initial free space range is added (e.g. during relocation).
Assign cache->space_info before the btrfs_add_new_free_space() call.
Reviewed-by: Boris Burkov <boris@bur.io>
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Pull btrfs fixes from David Sterba:
- preserve inode compression level when changing attributes
- fix lost wakeup when waiting for a zstd workspace
- fix bio context leaks after ordered extent processing errors
- in send, handle unexpected extents for non-regular inodes
- handle edge case in creation of reloc tree with enabled quotas
- in scrub report the exact failing offset, not the stripe base
- error handling fixes
- error code propagation in send, zoned mode and raid-stripe-tree
- restore active device pointer after seeding device addition error
- transaction abort fixups
- update Chris' email address
* tag 'for-7.3-rc1-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux:
MAINTAINERS: update Chris Mason's email address
btrfs: tests: do not touch page cache if root/inode allocation failed
btrfs: zstd: fix lost wakeup when waiting for a workspace
btrfs: do not force reloc root creation during qgroup_account_snapshot()
btrfs: send: fix lost error return value in will_overwrite_ref()
btrfs: abort transaction before releasing tree_log_mutex on commit failure
btrfs: zoned: propagate do_zone_finish() error in btrfs_zone_finish_endio()
btrfs: zoned: finish active block group cleanup if call_zone_finish() fails
btrfs: send: reject extents for non-regular inodes
btrfs: return proper negative error code for update_raid_extent_item()
btrfs: fix the possible bioc_list memory leak during error
btrfs: fix transaction use-after-free in raid stripe insertion
btrfs: scrub: report the failing sector's address, not the stripe base
btrfs: preserve the compression property when other inode flags change
btrfs: restore active device pointers after failed sprout
btrfs: detach failed sprout device from transaction update list
btrfs: clean up target device if block group marking fails
|
|
Inside test_find_delalloc() of extent-io-tests.c, if we fail to allocate
a dummy root or the test inode, we go to out label to clean up.
But at that stage, @inode is still NULL and we will call
process_page_range() to access the page cache of the inode, this will
cause NULL pointer dereference.
This is a very minor bug, as it only affects selftests which are not
compiled in by default for most distros, and very hard to trigger.
Fix it by adding a new out_root_info label to handle root and inode
allocation failure.
This is a pre-existing bug reported by Sashiko while reviewing another
patch.
Link: https://sashiko.dev/#/patchset/cover.1786095309.git.wqu%40suse.com
Reviewed-by: Boris Burkov <boris@bur.io>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
A writer can sleep forever in zstd_get_workspace() even though a workspace
is free. When zstd_alloc_workspace() fails, the task is queued on
zwsm->wait and schedules unconditionally, never re-testing the pool.
zstd_put_workspace() publishes the workspace and then calls cond_wake_up(),
which only wakes when a sleeper is already visible, so a workspace returned
between the failed allocation and prepare_to_wait() wakes nobody. The
window is wide: zstd_alloc_workspace() goes through kvmalloc() and may
enter reclaim.
Only a max level workspace triggers the wakeup and one is deliberately kept
allocated as the fallback every waiter waits for, so once its wakeup is
lost the writer stays in TASK_UNINTERRUPTIBLE until some other task happens
to return one. Re-check the pool after prepare_to_wait() has published the
waiter, and use the workspace if one turned up.
Fixes: 3f93aef535c8 ("btrfs: add zstd compression level support")
Assisted-by: Claude:claude-opus-5
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: FAN YE <fy15309206903@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
[BUG]
When running btrfs/252 with quota enabled through MKFS_OPTIONS="-O quota",
it has a high chance to trigger the following kernel warning and flips
the fs RO:
BTRFS info (device dm-2): relocating block group 30408704 flags metadata|dup
------------[ cut here ]------------
WARNING: fs/btrfs/extent-tree.c:879 at lookup_inline_extent_backref+0x74b/0x960 [btrfs], CPU#4: btrfs/2173
CPU: 4 UID: 0 PID: 2173 Comm: btrfs Not tainted 7.2.0-rc6-custom+ #457 PREEMPT(full) 3adc6528fb66f7a55fe1095385818e742f200aab
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS unknown 02/02/2022
RIP: 0010:lookup_inline_extent_backref+0x74b/0x960 [btrfs]
Call Trace:
<TASK>
insert_inline_extent_backref+0x7c/0x160 [btrfs 32f09462c54d9c922fca74a3e4866f4aa7737b72]
__btrfs_inc_extent_ref+0xa9/0x270 [btrfs 32f09462c54d9c922fca74a3e4866f4aa7737b72]
__btrfs_run_delayed_refs+0x4af/0x11c0 [btrfs 32f09462c54d9c922fca74a3e4866f4aa7737b72]
btrfs_run_delayed_refs+0x9d/0xf0 [btrfs 32f09462c54d9c922fca74a3e4866f4aa7737b72]
create_pending_snapshot+0x39d/0xf00 [btrfs 32f09462c54d9c922fca74a3e4866f4aa7737b72]
create_pending_snapshots+0x9b/0xc0 [btrfs 32f09462c54d9c922fca74a3e4866f4aa7737b72]
btrfs_commit_transaction+0x280/0xeb0 [btrfs 32f09462c54d9c922fca74a3e4866f4aa7737b72]
prepare_to_relocate+0x147/0x200 [btrfs 32f09462c54d9c922fca74a3e4866f4aa7737b72]
relocate_block_group+0x6b/0x5e0 [btrfs 32f09462c54d9c922fca74a3e4866f4aa7737b72]
btrfs_relocate_block_group+0x92c/0x2380 [btrfs 32f09462c54d9c922fca74a3e4866f4aa7737b72]
btrfs_relocate_chunk+0x3f/0x1a0 [btrfs 32f09462c54d9c922fca74a3e4866f4aa7737b72]
btrfs_balance+0xa2c/0x19c0 [btrfs 32f09462c54d9c922fca74a3e4866f4aa7737b72]
btrfs_ioctl+0x2839/0x2d30 [btrfs 32f09462c54d9c922fca74a3e4866f4aa7737b72]
__x64_sys_ioctl+0x416/0x9a0
do_syscall_64+0xe1/0x790
entry_SYSCALL_64_after_hwframe+0x4b/0x53
</TASK>
---[ end trace 0000000000000000 ]---
BTRFS info (device dm-2): leaf 4593991680 gen 233 total ptrs 175 free space 5953 owner 2
BTRFS info (device dm-2): refs 3 lock_owner 2173 current 2173
item 0 key (166772736 METADATA_ITEM 1) itemoff 16250 itemsize 33
extent refs 1 gen 222 flags 2
ref#0: tree block backref root 266
[ Skip the tree dump ]
item 174 key (263225344 METADATA_ITEM 0) itemoff 10328 itemsize 33
extent refs 1 gen 162 flags 258
ref#0: tree block backref root 267
BTRFS error (device dm-2): extent item not found for insert, bytenr 179847168 num_bytes 16384 parent 4594335744 root_objectid 273 owner 0 offset 0
BTRFS error (device dm-2): failed to run delayed ref for logical 179847168 num_bytes 16384 type 182 action 1 ref_mod 1: -117
[CAUSE]
The above error is showing that there is a tree reference to a metadata
extent that is no longer there.
With "ref_verify" mount option (requires CONFIG_BTRFS_DEBUG), there is
some extra debug output:
BTRFS error (device dm-2): dumping block entry [180961280 16384], num_refs 0, metadata 1, from disk 0
BTRFS error (device dm-2): root entry 256, num_refs 18446744073709551615
BTRFS error (device dm-2): root entry 273, num_refs 18446744073709551615
BTRFS error (device dm-2): Ref action 3, root 273, ref_root 273, parent 0, owner 0, offset 0, num_refs 1
btrfs_force_cow_block+0x129/0x7d0 [btrfs]
btrfs_cow_block+0x10a/0x250 [btrfs]
btrfs_search_slot+0x5eb/0xf40 [btrfs]
btrfs_insert_empty_items+0x3a/0x70 [btrfs]
insert_with_overflow+0x53/0x130 [btrfs]
btrfs_insert_dir_item+0x125/0x290 [btrfs]
btrfs_add_link+0xaa/0x410 [btrfs]
btrfs_rename+0x5ea/0xcd0 [btrfs]
btrfs_rename2+0x28/0x60 [btrfs]
vfs_rename+0x5b2/0xe10
filename_renameat2+0x244/0x430
__x64_sys_rename+0x48/0x70
do_syscall_64+0xe1/0x790
entry_SYSCALL_64_after_hwframe+0x4b/0x53
BTRFS error (device dm-2): Ref action 2, root 273, ref_root 273, parent 0, owner 0, offset 0, num_refs 18446744073709551615
btrfs_force_cow_block+0x327/0x7d0 [btrfs]
btrfs_cow_block+0x10a/0x250 [btrfs]
btrfs_search_slot+0x5eb/0xf40 [btrfs]
btrfs_lookup_file_extent+0x4d/0x70 [btrfs]
btrfs_drop_extents+0x151/0xf00 [btrfs]
insert_reserved_file_extent+0xfe/0x3e0 [btrfs]
btrfs_finish_one_ordered+0x549/0xc40 [btrfs]
btrfs_work_helper+0xde/0x350 [btrfs]
process_one_work+0x198/0x380
worker_thread+0x1c8/0x330
kthread+0xee/0x120
ret_from_fork+0x28f/0x310
ret_from_fork_asm+0x11/0x20
BTRFS error (device dm-2): Ref action 1, root 273, ref_root 0, parent 4594335744, owner 0, offset 0, num_refs 1
__btrfs_mod_ref+0x1c5/0x2d0 [btrfs]
btrfs_copy_root+0x262/0x390 [btrfs]
create_reloc_root+0xb9/0x370 [btrfs]
btrfs_init_reloc_root+0xb0/0x1b0 [btrfs]
record_root_in_trans+0xa6/0xd0 [btrfs]
create_pending_snapshot+0x383/0xf00 [btrfs]
create_pending_snapshots+0x9b/0xc0 [btrfs]
btrfs_commit_transaction+0x280/0xeb0 [btrfs]
prepare_to_relocate+0x147/0x200 [btrfs]
relocate_block_group+0x6b/0x5e0 [btrfs]
btrfs_relocate_block_group+0x92c/0x2380 [btrfs]
btrfs_relocate_chunk+0x3f/0x1a0 [btrfs]
btrfs_balance+0xa2c/0x19c0 [btrfs]
btrfs_ioctl+0x2839/0x2d30 [btrfs]
__x64_sys_ioctl+0x416/0x9a0
do_syscall_64+0xe1/0x790
The above shows the direct cause, Ref action 3 is the oldest operation,
which shows the tree block is created by COW. Then ref action 2 shows
it's COWed away, by a metadata update, meaning the tree block is already
released, should not be referred any more.
Then the final one, is trying to create a reloc tree for subvolume 273,
and that reloc root creation is referring to the already dropped tree
block.
The root cause is that, during qgroup_account_snapshot(), we are calling
record_root_in_trans() with "force = true".
So if the root has no reloc root, we will create one, but at that
timing it's already too late.
Normally reloc root should be created before the commit and current
roots diverge, to avoid the same problem we are hitting.
But during relocation initialization, we are committing the current
running transaction, with a new reloc_control attached halfway.
And if qgroup is enabled, the record_root_in_trans() with "force = true"
calls will force reloc root creation even if we do not and should not
create reloc root at that timing.
[FIX]
Do not force reloc root creation during record_root_in_trans() with
"force = true" cases, which is only called by qgroup_account_snapshot().
If we're really under relocation, the reloc root should be created way
early, before the commit and current root diverge. If the root has no
reloc tree yet, it means we're still initializing the reloc, and do not
need a reloc root.
So skipping the reloc tree creation in qgroup_account_snapshot() should
be safe.
Link: https://bugzilla.suse.com/show_bug.cgi?id=1275740
Fixes: 4d31778aa2fa ("btrfs: qgroup: Fix root item corruption when multiple same source snapshots are created with quota enabled")
Assisted-by: LLM (initial analysis, but incorrect conclusion with too many burnt tokens)
Tested-by: Disha Goel <disgoel@linux.ibm.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
The direct-return refactoring in commit b3047a42f55d ("btrfs: send:
directly return from will_overwrite_ref() and simplify it") changed
will_overwrite_ref() to return directly instead of going through the
common out label.
That resulted in a negative return value from is_inode_existent() to
start being converted to 0, making lookup errors unable to be
distinguished from the inode not existing.
process_recorded_refs() expects negative errors from
will_overwrite_ref() and aborts processing when it receives one.
Return the value from is_inode_existent() to restore the previous error
propagation behavior as it was before the refactor.
Fixes: b3047a42f55d ("btrfs: send: directly return from will_overwrite_ref() and simplify it")
Signed-off-by: Avi Weiss <thnkslprpt@gmail.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
When transaction metadata writeout fails in btrfs_commit_transaction(),
the current code only logs the error, drops tree_log_mutex and then goes
through cleanup_transaction(), which aborts the transaction and records
the fs error.
That is too late for the tree log side. A log sync can already be
waiting on tree_log_mutex, because the committing transaction is moved
to TRANS_STATE_UNBLOCKED while that mutex is held, which lets fsyncs
join the next transaction and queue up in btrfs_sync_log(). Once the
failed commit drops tree_log_mutex, such a log sync acquires it, sees
BTRFS_FS_ERROR() still clear, and writes super_for_commit. That
superblock holds the roots prepared for the transaction that has just
failed to write out its metadata, so it can point at tree blocks that
never reached the disk, and the next mount fails with a parent transid
mismatch.
Commit 165ea85f1483 ("btrfs: do not write supers if we have an fs
error") fixed this class of problem by making btrfs_sync_log() check for
an fs error right after taking tree_log_mutex. That check only works if
the commit path publishes the fs error before it releases the same
mutex, and commit 68d4ece9c30e ("btrfs: don't call
btrfs_handle_fs_error() in btrfs_commit_transaction()") removed the only
thing that did so.
Restore the ordering by aborting the transaction while tree_log_mutex is
still held. We have a transaction handle here, so this does not need to
bring back the btrfs_handle_fs_error() call: __btrfs_abort_transaction()
records the fs error itself, which is all btrfs_sync_log() looks at, and
the error message put in its place is kept.
This is what commit 3810ab40afa5 ("btrfs: abort transaction on error in
write_all_supers()") already does for the next call in this function.
This is reproducible on an unmodified kernel by failing the first
couple of bios of a transaction commit with fail_make_request while a
concurrent fsync workload keeps log syncs queued on tree_log_mutex.
Fixes: 68d4ece9c30e ("btrfs: don't call btrfs_handle_fs_error() in btrfs_commit_transaction()")
CC: stable@vger.kernel.org # 7.0+
Reviewed-by: Boris Burkov <boris@bur.io>
Reviewed-by: jlayton@meta.com <jlayton@meta.com>
Signed-off-by: Leo Martins <loemra.dev@gmail.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
btrfs_zone_finish_endio() ignored the return value of do_zone_finish()
and always returned 0, silently dropping a failed zone finish.
Instead propagate any error from do_zone_finish() as the caller
btrfs_finish_ordered_io() already handles it.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
do_zone_finish() clears BLOCK_GROUP_FLAG_ZONE_IS_ACTIVE before finishing
the zones. If call_zone_finish() then fails it returned early, leaving the
now inactive block group on fs_info->zone_active_bgs, leaking its
reference, the BTRFS_FS_NEED_ZONE_FINISH waiters are never woken, and as
its alloc_offset equals the zone capacity btrfs_zone_finish_one_bg() keeps
selecting it, spinning btrfs_zoned_activate_one_bg().
Fall through to the cleanup on failure too and return the error, but keep
the block group read-only as its zones are left inconsistent.
Fixes: d70cbdda75da ("btrfs: zoned: consolidate zone finish functions")
Link: https://sashiko.dev/#/patchset/20260818100037.1366563-1-johannes.thumshirn%40wdc.com
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
[BUG]
A corrupted subvolume tree can leave an EXTENT_DATA item attached to an
inode whose mode is not S_IFREG or S_IFLNK. During send, such an item can
be treated as file data and crash through a NULL address_space operation:
BUG: kernel NULL pointer dereference, address: 0000000000000000
#PF: supervisor instruction fetch in kernel mode
#PF: error_code(0x0010) - not-present page
Call Trace:
<TASK>
read_pages+0x80b/0xb30 mm/readahead.c:173
page_cache_ra_unbounded+0x40d/0x890 mm/readahead.c:302
do_page_cache_ra mm/readahead.c:332 [inline]
page_cache_ra_order+0xa16/0xcd0 mm/readahead.c:535
page_cache_sync_ra+0x5ce/0x9d0 mm/readahead.c:626
page_cache_sync_readahead include/linux/pagemap.h:1379 [inline]
put_file_data fs/btrfs/send.c:5224 [inline]
send_write fs/btrfs/send.c:5291 [inline]
send_extent_data+0x16b2/0x29b0 fs/btrfs/send.c:5715
send_write_or_clone fs/btrfs/send.c:6135 [inline]
process_extent+0x5d4/0x17b0 fs/btrfs/send.c:6504
changed_extent fs/btrfs/send.c:7079 [inline]
changed_cb+0x22f9/0x3cd0 fs/btrfs/send.c:7245
full_send_tree fs/btrfs/send.c:7318 [inline]
send_subvol fs/btrfs/send.c:7910 [inline]
btrfs_ioctl_send+0x46a9/0x57f0 fs/btrfs/send.c:8248
...
[CAUSE]
process_extent() skips extent items for symlinks but assumes every other
inode with an extent item is a regular file. For a corrupted non-regular
inode, btrfs_iget() does not install the regular file address_space
operations. The readahead fallback can then call a NULL read_folio
callback before the existing validation in btrfs_get_extent() can run.
[FIX]
Reject extent items for inode types other than regular files and symlinks
at the common send extent-processing boundary. Symlink handling is left
unchanged because send emits symlink data from read_symlink(). This covers
full, incremental and new-generation sends without adding a check to the
regular I/O path.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: ZhengYuan Huang <gality369@gmail.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
The function btrfs_abort_transaction() only accepts negative error code,
and have the macro VERIFY_NEGATIVE_ERROR() to verify that error code.
But inside update_raid_extent_item(), if there is such key found, we
return 1, breaking the negative error code scheme.
Furthermore if we hit some real error during the tree search, e.g. -EIO,
then the error code is always over-written to -EINVAL.
Fix both problems by following other call sites by overwriting @ret to
-ENOENT if the btrfs_search_slot() failed to locate the key.
This is very unlikely to hit, as we only enter update_raid_extent_item()
if there is a conflicting key already in the raid stripe tree.
This was reported by Sashiko when reviewing another patch.
Link: https://sashiko.dev/#/patchset/20260817021512.3010812-1-shuangpeng.kernel%40gmail.com
Fixes: 8c4cba2adbb0 ("btrfs: update stripe extents for existing logical addresses")
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
There are two possible ways to leak bioc memory on
btrfs_ordered_extent::bioc_list:
- An error occurred for btrfs_insert_one_raid_extent()
Then the function btrfs_insert_raid_extent() immediately return
without freeing any bioc in the bioc_list.
- An ordered extent hit an IO error
In that case the ordered extent will have BTRFS_ORDERED_IOERR set, and
skip the call on btrfs_insert_raid_extent() completely.
Fix the problem by:
- Introduce a new helper, btrfs_cleanup_ordered_bioc_list()
Which will remove all bioc from the bioc_list, and release the bioc.
- Call the above helper for btrfs_insert_raid_extent()
So that the cleanup helper is always called no matter what.
- Call the above helper for btrfs_finish_one_ordered()
This is called just before the final release on the ordered extent.
This was reported by Sashiko when reviewing another patch.
Link: https://sashiko.dev/#/patchset/20260817021512.3010812-1-shuangpeng.kernel%40gmail.com
Fixes: 02c372e1f016 ("btrfs: add support for inserting raid stripe extents")
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
If allocation of a RAID stripe extent fails,
btrfs_insert_one_raid_extent() aborts and ends the transaction before
returning -ENOMEM.
btrfs_finish_one_ordered(), the production caller through
btrfs_insert_raid_extent(), still owns the transaction handle. It handles
the error by aborting the transaction and then reaches the common exit
path, which ends the transaction again.
The premature end can free the handle and drop its transaction reference.
Transaction cleanup can then free the transaction before the caller's
second abort accesses the handle and transaction, resulting in
use-after-free.
Keep the abort at the failure site, but let the caller's common exit path
end the transaction once, after it has finished using both objects.
Fixes: 02c372e1f016 ("btrfs: add support for inserting raid stripe extents")
Assisted-by: Codex:GPT-5
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Shuangpeng Bai <shuangpeng.kernel@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
scrub_stripe_report_errors() iterates over the sectors of a stripe, but
every message it emits passes stripe->logical, the address of the first
sector of the 64KiB stripe, rather than the address of the sector being
reported. The physical address is likewise computed once, before the
loop, from stripe->logical.
This matters because scrub_print_common_warning() uses that logical
address for the backref walk which produces the "root %llu inode %llu
offset %llu ... (path: ...)" part of the message. As the address is
always the stripe base, the reported root/inode/offset/path can identify
a different file from the one whose sector actually failed.
A 64KiB stripe routinely spans several extents belonging to unrelated
files. On the machine where this was found, the stripe at logical
0x17D9380000 holds four sectors of /usr/share/plasma/emoji/bg.dict, then
a file inside a docker volume, then sectors referenced only by
snapshots. Every error anywhere in that stripe is attributed to bg.dict.
The effect is visible statistically: across ten months and four kernel
series that machine logged 81 distinct flagged logical addresses, and
every one of them is exactly 64KiB aligned. Since BTRFS_STRIPE_LEN is
64KiB and stripe->logical is stripe aligned by construction, real
failures distributed across sectors could not produce that.
Report the address of the sector actually being examined. Adding the
sector offset to the physical address is valid because BTRFS_STRIPE_LEN
is the unit contiguous on a single device for every profile, so a stripe
never crosses a device boundary.
Fixes: 0096580713ff ("btrfs: scrub: introduce error reporting functionality for scrub_stripe")
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: James C. Owens <jamesowens@optonline.net>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Setting the compression property on an inode also sets BTRFS_INODE_COMPRESS
on it, and btrfs_inode_flags_to_fsflags() reports that back as FS_COMPR_FL
to FS_IOC_GETFLAGS. chattr(1), like any other FS_IOC_SETFLAGS caller, reads
the current flags, flips only the bit the user asked for and writes the
whole set back, so a request as unrelated as "chattr +i" reaches
btrfs_fileattr_set() with FS_COMPR_FL set.
btrfs_fileattr_set() takes that as a request to enable compression and
overwrites the compression property with the algorithm from the mount
options, falling back to zlib when the filesystem was not mounted with
-o compress. The algorithm the user selected is silently replaced:
# btrfs property set /mnt/foo compression zstd
# btrfs property get /mnt/foo compression
compression=zstd
# chattr +i /mnt/foo
# btrfs property get /mnt/foo compression
compression=zlib
Every chattr operation triggers this, not just +i, and directories are
affected as well, so files created afterwards inherit the wrong algorithm
too. On a filesystem mounted with -o compress=lzo the property is replaced
with lzo instead. Recovering needs a chattr -i first, because the immutable
flag rejects the setxattr that "btrfs property set" issues.
Prefer the algorithm recorded in the compression property and only fall
back to the mount default when there is no property, so that unrelated
flag changes no longer overwrite the user's choice. Inodes that have the
compress flag set but no property still get the default, so they behave
as before.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Sam Ho <samho@synology.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
btrfs_init_new_device() switches latest_dev and possibly s_bdev from the
seed device to the new sprout device before creating the first writable
chunks.
If chunk creation or the subsequent sprout setup fails, the error path
releases the new device without switching those pointers back.
btrfs_show_devname() can then dereference the freed latest_dev and crash.
Restore the active device pointers to the latest seed device before
removing and releasing the failed sprout device.
Fixes: b7cb29e666fe ("btrfs: update latest_dev when we create a sprout device")
Assisted-by: Codex:gpt-5
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Guanghui Yang <3497809730@qq.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
When creating the first metadata chunk for a sprout filesystem,
create_chunk() adds the new device to the transaction dev_update_list
through device->post_commit_list.
If the subsequent system chunk creation fails, btrfs_init_new_device()
aborts the transaction and releases the device while post_commit_list is
still linked. This triggers a warning in btrfs_free_device() and leaves
the transaction list referencing freed memory.
Detach the device while holding chunk_mutex before releasing it.
Fixes: bbbf7243d62d ("btrfs: combine device update operations during transaction commit")
Assisted-by: Codex:gpt-5
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Guanghui Yang <3497809730@qq.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
btrfs_dev_replace_start() adds the replacement target to the device list
before marking block groups to copy. If marking fails, returning
directly leaves the target linked and keeps the device accounting
incremented.
Jump to the existing cleanup path so the target device is removed and
released on failure.
The issue was found by a failure-path metadata residual analyzer and
verified with targeted failure injection on v6.14.
Assisted-by: Codex:gpt-5
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Guanghui Yang <3497809730@qq.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Pull block updates from Jens Axboe:
- NVMe updates via Keith:
- Enable Clang context analysis for the nvme host driver, adding
context annotations across core, fabrics, rdma, tcp and pci
- nvmet reservation state exposed through a new namespace-level
debugfs directory, plus ABI documentation for the host sysfs and
target configfs interfaces
- nvme-tcp host memory disclosure fixes on the read path: reject a
read that transferred too few bytes, don't accept C2HData based
on blk_rq_payload_bytes() alone, and fix the R2T case for a read
command
- Parallelize nvme-rdma I/O queue allocation and startup (Surabhi)
- Apple nvme fixes and quirks: page aligned admin queue buffers,
destroy the admin queue on removal, and various DMA/NVMMU
correctness fixes
- A large pile of nvmet and host fixes for out-of-bounds reads,
refcount/resource leaks, and NULL derefs across auth, zns,
passthru, pci-epf, rdma and configfs
- Various other fixes and cleanups
- MD updates via Yu Kuai:
- llbitmap reshape support, the large series wiring exact bitmap
mapping and reshape lifecycle through raid5 and raid10, growing
the page cache in place, and remapping checkpointed bits as
reshape progresses
- raid5 fixes for lockless max_nr_stripes and recovery_offset
accesses, a reshape deadlock with more failed devices than max
degraded, and bitmap batch counter consistency
- Atomic write handling for raid1/raid10, and removal of the
REQ_NOWAIT support from raid1/10/456
- raid5-ppl use-after-free fix in ppl_do_flush()
- A batch of smaller fixes across md core and the bitmap code
- s390/dasd ESE full-track write support and the surrounding
infrastructure, plus enabling CONTEXT_ANALYSIS for s390/block
- RWF_DONTCACHE support for block devices, built on new task-context
bio completion infrastructure, and wiring it up for the iomap and
buffer dropbehind writeback paths
- Async io_uring zone reset all, plus zone management command cleanups
allowing REQ_NOWAIT and tightening conventional zone rejection
- Block integrity refactoring: lift BIP_CHECK_FLAGS to the shared
header, handle nogenerate/noverify properly in fs-integrity, and drop
the blk-integrity.h include from bdev.c
- Split out a new blk_plug.h header
- ublk improvements: add UBLK_F_IO_DESC_SIZE, split request validation
from io_desc init, reject non-power-of-2 zone sizes in SET_PARAMS,
and a series of hardening fixes around map/unmap and auto buf reg
- null_blk cleanups and configfs serialization fixes
- nbd queue freeze removal on the setup paths, and a new
pre_defined_connections module parameter for pre-created devices
- blk-cgroup fixes for the race between policy activation and blkg
destruction, and accounting per-cpu stats over possible CPUs across
blk-stat, iolatency, iocost and kyber
- Various dio fixes: leak on metadata mapping error, validate user
space vectors during extraction, and set dma_alignment from the
backing file for loop and zloop direct I/O
- bio cleanups
- Various other fixes and cleanups all over
* tag 'for-7.3/block-20260819' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux: (241 commits)
nbd: add pre_defined_connections module parameter for pre-created devices
nbd: remove queue freeze for newly created nbd from netlink path
nbd: factor out a nbd_genl_foreach_sock
nbd: skip queue freeze when setting size at device startup
nbd: remove queue freeze in nbd_add_socket
nbd: clear queue limits on disconnect
nbd: disallow NBD_SET_SOCK on an active device
nbd: simplify find_fallback() by removing redundant logic
blk-mq: add missing call to srcu_barrier() in blk_mq_free_tag_set()
block: mtip32xx: synchronize ioctls with device removal
ublk: avoid teardown retry loop on xarray allocation failure
null_blk: fix UBSAN shift-out-of-bounds when zone_size is 0 or overflows
block: don't include blk-integrity.h in bdev.c
xfs: avoid double deferrals for RWF_DONTCACHE writes
loop: Fix recently introduced lock inversion
block: set QUEUE_FLAG_DYING unconditionally in blk_mark_disk_dead()
swim3: Add missing MODULE_DESCRIPTION
selftests: ublk: add SET_PARAMS validation test
selftests: ublk: add helper for SET_PARAMS
ublk: reject non-power-of-2 zone sizes in SET_PARAMS
...
|
|
Pull btrfs updates from David Sterba:
"This is the summer edition of btrfs changes, smaller than usual. Yet,
there are performance improvements in various areas or for specific
workloads and some notable changes like removing space cache v1 code
or mount option reduction.
User visible changes:
- free space v1 disabled by default; the v2 (free space tree) is mkfs
default since 5.15, filesystems with v1 still work but could be
slightly slower due to lack of block group caching
- mount option 'rescue=usebackuproot' requires read-only mount, it's
too risky to allow writable mount
- remove standalone mount option 'usebackuproot', deprecated in 5.9
- preserve constraints of NODATASUM and NODATACOW when chattr and
mount options may change the attributes
- remove arbitrary limitation of 4KiB for page size when allowing
block sizes smaller than page
- print messages when pinned block groups affect swap activation
Performance improvements:
- use iomap bounce buffer for direct io instead of a fall back to
buffered io; past correctness vs speed trade-offs dropped
performance to ~50% of theoretical maximum, now it's ~95%,
effectively doubled
- replace xarray with local LRU list for tracking inhibited
extent buffers, restored performance to pre-inhibition state
(relatively ~3x)
- remove unnecessary 1 jiffy delay in "non-SSD" mode with multiple
logging tasks, decrease latency, throughput increased ~5x on sample
workload
- skip hole detection during full fsync for files without holes
and lots of extents, reduce run time ~5x on sample workload
(microsecond ranges)
- reduce locking around extent readahead so it does not slow down
other tasks using an overlapping range
- enhance extent buffer allocation modes to allow NOWAIT semantics
in some cases
Notable fixes:
- write-protect folios during writeback, prevent concurrent mmap
and compress/checksumming/etc undesired interactions
- in zoned mode, handle transient overcommit full instead of going
read-only
- fix possible deadlock between defragmentation and delayed
allocation reservations
- handle remaining iputs at umount time
- fix lockdep warning between device scan locking and log mutex
- add workaround for degenerate RAID56 device count modes (2 and 3)
not supported by the parity calculation library
- restore check that subvolume is not read-only when changing ACLs
- retry reading verity data colliding with up-to-date status changes
Core:
- simplify raid56 stripe handling by using contiguous virtual
allocations
- in zoned mode, fix various metadata write issues in writeback or
unmount
- space reservation fixes
- remove unused data structure members
- more auto-freeing conversions
- error pointer values are printed using %pe format"
* tag 'for-7.3-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux: (72 commits)
btrfs: skip hole detection during full fsync for files without holes
btrfs: add extra ASSERT()s to make sure the folio size is correct
btrfs: use GFP_NOWAIT for tree block readahead
btrfs: enable unlocked NOFAIL retry for eb allocations
btrfs: add struct btrfs_eb_prealloc
btrfs: factor init_extent_buffer from __alloc_extent_buffer
btrfs: qgroup: fix a wrong length calculation in qgroup_free_reserved_data()
btrfs: add validation for extent states
btrfs: use aligned range for locking in reflink
btrfs: use aligned range for locking in extent_fiemap()
btrfs: zoned: don't clobber the extent buffer when zeroing it out
btrfs: zoned: drop stranded dirty metadata buffers at unmount
btrfs: zoned: drop stranded dirty metadata on transaction abort
btrfs: zoned: flush active metadata block group at btree_writepages() start
btrfs: convert reflink.c to use btrfs_inode as parameters
btrfs: use simple booleans for log_commit field in struct btrfs_root
btrfs: check for exit condition after waking in wait_log_commit()
btrfs: move condition for log commit wait into wait_log_commit()
btrfs: remove log batch counter use for fsync
btrfs: stop sleeping for one jiffy in non-ssd mounts during log commit
...
|
|
Pull vfs superblock updates from Christian Brauner:
- Make it possible to share a block device between multiple
filesystems.
erofs can mount read-only blob devices shared between many
superblocks, but because we only tracked a single superblock a
freeze, thaw, removal or sync on such a device was never propagated
to all the superblocks using it, and there was no way to find them.
Add an efficient table to lookup all superblocks using a given block
device.
- A bunch of pre-existing fixes fell out of this work:
A block-device freeze racing a btrfs device change could leave the
whole filesystem stuck frozen. A bdev_freeze() issued by "dmsetup
suspend" or an LVM snapshot resolves that holder to freeze the
filesystem. and bdev_thaw() resolves it again to thaw. A freeze
landing while btrfs is adding, removing or replacing a device freezes
the filesystem. The membership change then drops that link. So the
matching thaw could no longer find the superblock.
Forbid freezing a device for the duration of a membership change,
modelled on deny_write_access()/allow_write_access().
* tag 'vfs-7.3-rc1.super' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs: (24 commits)
super: fix dying superblock warning messages
block: reject block device inodes with i_rdev == 0 in lookup_bdev()
selftests/filesystems: add ustat() coverage
fs: look up the superblock via the device table in user_get_super()
super: make fs_holder_ops private
f2fs: open via dedicated fs bdev helpers
erofs: open via dedicated fs bdev helpers
fs: tolerate per-superblock freeze errors on shared devices
fs: look up superblocks via the device table in fs_holder_ops
ext4: open via dedicated fs bdev helpers
btrfs: open via dedicated fs bdev helpers
xfs: port to fs_bdev_file_open_by_path()
fs: add dedicated block device open helpers for filesystems
fs: maintain a global device-to-superblock table
ocfs2: don't reset s_dev on dismount
ext4: use anonymous devices for KUnit test superblocks
fs, block: move blk_mode_t and fop_flags_t into <linux/types.h>
super: take lock after last reference count
super: convert s_count to refcount_t s_passive
btrfs: deny freezing devices undergoing a replace
...
|
|
Pull misc vfs updates from Christian Brauner:
"Bigger cleanups:
- The lockref dead-count handling is tidied up.
The open-coded check for a count below zero as the dead marker
relies on information the caller should not have.
- make put_mnt_ns() leave mounts connected. Destroying a mount
namespace disconnected its mounts from their mount points. So a
file descriptor still open on the parent of a mount point could be
used to peek under it.
Locked mounts were already kept connected to prevent exactly that.
But a mount is only locked when its tree is copied across a user
namespace boundary. So a mount namespace set up by a privileged
component had no locked mounts and its mounts were disconnected.
Passing UMOUNT_CONNECTED keeps every mount connected and prevents
that bug.
- vfs_prepare_mode() passes S_IFDIR for directories. I meant to fix
that ago but didn't get to it. So now someone finally did it.
This kills the exception where the mode could be 0 when a directory
was created whereas every other creation operation passed it
explicitly already.
- move long delayed work for ufs, jffs2, hfsplus, hfs and affs from
the per-cpu system_long_wq to the new unbound system_dfl_long_wq.
None of that work relies on per-cpu state and the work item is
enqueued with queue_delayed_work() whose timer is global anyway. So
it may as well benefit from scheduler task placement.
Smaller fixes and cleanups:
- unlock_buffer() and journal_end_buffer_io_sync() use
clear_and_wake_up_bit()
- the pipe page pools are unified into a single per-pipe pool and the
extra wake_up(rd_wait) is limited to EPOLLET consumers
- eventpoll now computes its timer slack lazily in ep_poll()
- shrink_dcache_for_umount() keeps making progress on busy roots
- excess xarray nodes are freed in clear_inode()
- romfs detects hard link cycles
- the user path of nested backing files is fixed
- pidfd holds exec_update_lock around the namespace ioctl
- non-memcg-aware nr_cached_objects is skipped during memcg slab
shrink
- iomap_write_iter() always returns status
- mangle_path() is renamed to seq_mangle_path()
- inode timestamp accessors are annotated
- new regression test for pipe->poll_usage.
- a few documentation, kernel-doc and selftest fixes"
* tag 'vfs-7.3-rc1.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs: (67 commits)
selftests/namespaces: Fix racy pipe handshake in timens and pidns_separate
selftests/epoll: add a regression test for pipe->poll_usage
pipe: only enable the extra wake_up(rd_wait) for EPOLLET consumers
pidfd: hold exec_update_lock around namespace ioctl
fs: fix user path of nested backing files
fs: remove stale inode_insert5() kernel-doc parameter
fs: fix switch/case indentation in sysfs() syscall
fs: document semantics of kstat::{uid,gid} fields
dcache: keep shrink_dcache_for_umount() making progress on busy roots
seq_file: rename mangle_path to seq_mangle_path
nstree: add/fix struct ns_id_req kernel-doc member fields
dcache: use lockref routines for dead count checks
lockref: tidy up dead count handling
initramfs: fix typo in reserve_initrd_mem comment
fs/pipe: unify the page pools into a single per-pipe pool
fs: annotate inode timestamp accessors
eventpoll: compute timer slack lazily in ep_poll()
selftests/filesystems: add mntns cleanup test
put_mnt_ns(): leave mounts connected
affs: Move long delayed work on system_dfl_long_wq
...
|
|
Pull vfs lookup updates from Christian Brauner:
"This refactors lookup_open() and adds vfs_lookup_open() for nfsd.
mnt_want_write() and parent locking are moved into lookup_open()
itself.
audit_inode_child() is also now called in lookup_open() on failure.
That is the calling convention in vfs_create() and vfs_mkdir(), but
lookup_open() made no such call when atomic_open() should have created
a file and did not. And neither did the regular ->create() path fwiw.
This also contains work to remove the unneeded excl argument from the
->create() inode op"
* tag 'vfs-7.3-rc1.lookup' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs:
fs/namei.c: fix coding style in atomic_open() and lookup_open()
fs/namei.c: fix kerneldoc of atomic_open() and vfs_lookup_open()
fs/namei.c: update stale comments in lookup_open()
Remove excl arg to ->create inode_operation
fs/namei.c: update kerneldoc of atomic_open()
vfs: call audit_inode_child() in lookup_open() on failure
vfs: move create error && negative dentry case in lookup_open() up
VFS: add vfs_lookup_open() for nfsd
VFS: move delegated_inode retry loop into lookup_open()
VFS: move mnt_want_write() and locking into lookup_open()
|
|
Pull kthread vfs updates from Christian Brauner:
"This stops kernel threads from sharing filesystem state with
userspace. This work is about 3 cycles old and has been in -next
for about that time.
When the kernel boots init_task creates PID 1 and then kthreadd. From
that point every kthread and PID 1 share the same fs_struct. That is
why pivot_root() has to rewrite the fs_struct of all kthreads. The
rewriting exists so that kthreads can use init's filesystem state when
they want to. It also means userspace can move the ground out from
under the kernel.
PID 1 now gets a completely separate fs_struct. All kthreads are
anchored in a private SB_KERNMOUNT instance of nullfs that cannot be
mounted on and cannot be used to follow other mounts. Userspace init
can no longer affect kthread filesystem state and kthreads can no
longer affect userspace fs state without explicit opting in to that.
Path lookup from a kthread now fails by default. It makes it
deliberately hard to offload security sensitive operations into init's
filesystem state from a kthread.
Places that legitimately need to look something up there opt in
through the new scoped_with_init_fs() which temporarily overrides the
caller's fs_struct with init's. usermodehelpers remain the only kernel
tasks that genuinely share init's filesystem state, since they execute
random binaries in the root filesystem (excellent...).
The visible result is that /proc/2/root is a nullfs with an empty
mountinfo while /proc/1/root is the real root"
* tag 'vfs-7.3-rc1.kthread' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs: (26 commits)
initramfs_test: use test init/exit hooks to override init fs
fs: stop rewriting paths for PF_EXITING | PF_DUMPCORE
fs: stop rewriting kthread fs structs
fs: start all kthreads in nullfs
nullfs: make nullfs multi-instance
devtmpfs: create private mount namespace
fs: add umh argument to struct kernel_clone_args
fs: stop sharing fs_struct between init_task and pid 1
af_unix: use scoped_with_init_fs() for coredump socket lookup
initramfs: use scoped_with_init_fs() for rootfs unpacking
pnfs/blocklayout: use scoped_with_init_fs() for SCSI device lookup
ksmbd: use scoped_with_init_fs() for VFS path operations
ksmbd: use scoped_with_init_fs() for filesystem info path lookup
ksmbd: use scoped_with_init_fs() for share path resolution
fs: use scoped_with_init_fs() for kernel_read_file_from_path_initns()
coredump: use scoped_with_init_fs() for coredump path resolution
btrfs: use scoped_with_init_fs() for update_dev_time()
scsi: target: use scoped_with_init_fs() for APTPL metadata
scsi: target: use scoped_with_init_fs() for ALUA metadata
crypto: ccp: use scoped_with_init_fs() for SEV file access
...
|
|
Pull iomap updates from Christian Brauner:
"The bulk of this is the conversion of iomap to a single ->iomap_next()
callback and thus finishing the move to an iterator model.
Every iomap operation drove its iteration through a struct iomap_ops
holding ->iomap_begin() and ->iomap_end(). iomap_iter() only ever sees
those as pointers. That means every step of every iteration is an
indirect call.
This collapses both into one ->iomap_next() callback that finishes the
previous mapping and produces the next one. This lets callers inline
the iteration loop and pass its ->iomap_next() as a compile time
constant. That means the compiler can turn it into a direct and hence
inlineable call.
This also allows future callers to express custom logic to drive the
iteration forward better. xfs, btrfs, ext4, ext2, erofs, f2fs, gfs2,
hpfs, fuse, exfat, zonefs, ntfs, ntfs3 and the block device mapping
are all converted. No functional changes are intended.
This also adds a simple direct I/O path for small reads. On Gen5 NVMe
the __iomap_dio_rw() dominates 4K random reads. The same single-core
io_uring poll mode workload reaches ~3.2M IOPS against the raw block
device but only ~1.92M through ext4 or XFS.
__iomap_dio_rw(), iomap_iter(), iomap_dio_bio_iter() and kfree() were
at the top of the profile. The new path is very lightweight if no
special behavior is requested. The bio comes from a dedicated bioset
and laid out so the whole request is a single cacheline aligned
allocation. Completion runs inline.
That takes ext4 from 1.92M to 2.19M IOPS in the original workload. fio
shows around:
- 4% at libaio queue depths of 64 and up
- around 5% for io_uring
- up to 10% for io_uring poll mode at depth 256
on both ext4 and xfs.
A few other patches:
- iomap_folio_mark_uptodate() lets a filesystem that writes into the
page cache outside the iomap read and write paths keep iomap's
internal uptodate bitmap in sync, which fuse needs for
server-pushed notify stores before it can enable large folios;
- two fixes for iomap_bio_read_folio_range_sync(): a potential crash
when device integrity behavior is changed and a missing
bio_uninit().
- a folio batch release fix on iomap callback failures
- FGP_NOFS is dropped from iomap_get_folio()
- documentation fix"
* tag 'vfs-7.3-rc1.iomap' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs: (29 commits)
iomap: iomap_bio_read_folio_range_sync is missing a call to bio_uninit
iomap: don't free integrity payload that doesn't exist
docs: fix grammatical error in iomap docs
exfat: convert iomap ops to ->iomap_next()
fuse: convert iomap ops to ->iomap_next()
hpfs: convert iomap ops to ->iomap_next()
gfs2: convert iomap ops to ->iomap_next()
f2fs: convert iomap ops to ->iomap_next()
block: convert iomap ops to ->iomap_next()
ext2: convert iomap ops to ->iomap_next()
zonefs: convert iomap ops to ->iomap_next()
erofs: convert iomap ops to ->iomap_next()
ext4: convert iomap ops to ->iomap_next()
ntfs: convert iomap ops to ->iomap_next()
ntfs3: convert iomap ops to ->iomap_next()
btrfs: convert iomap ops to ->iomap_next()
xfs: convert iomap ops to ->iomap_next()
iomap: add ->iomap_next()
iomap: use GFP_NOWAIT when application for iomap_dio_simple allocations
iomap: decouple simple direct I/O reads from iomap_dio_rw
...
|
|
If we the no-holes feature is enabled (a default since btrfs-progs 5.15),
when doing a full fsync we always iterate of all leaves in the subvolume
root that contain file extent items in order to detect holes between them.
This can take a lot of time for files with a large number of extents.
But if we know there are no prealloc extents and the amount of space
(uncompressed space) is greater than or equals to the i_size of the
inode, then we cannot have holes and therefore avoid searching for
them. So skip the search if those conditions are met.
The following test script was used:
$ cat test.sh
#!/bin/bash
MNT=/mnt/nullb0
DEV=/dev/nullb0
umount $MNT &> /dev/null
mkfs.btrfs -f $DEV
mount $DEV $MNT
# 256M gives 64K extents of 4K each.
FILE_SIZE=$((256 * 1024 * 1024))
touch $MNT/foobar
for ((i = 0; i < $FILE_SIZE; i += 8192)); do
xfs_io -c "pwrite -S 0xab $i 4K" $MNT/foobar > /dev/null
done
xfs_io -c "fsync" $MNT/foobar
for ((i = 4096; i < $FILE_SIZE; i += 8192)); do
xfs_io -c "pwrite -S 0xab $i 4K" $MNT/foobar > /dev/null
done
# unmount and mount, clear caches and ensure the next fsync is a
# full sync.
umount $MNT
mount $DEV $MNT
# Do some change to the file in order to fsync.
xfs_io -c "pwrite -S 0xcd 0 4K" $MNT/foobar > /dev/null
T0=$(date +%s%N)
xfs_io -c "fsync" $MNT/foobar
T1=$(date +%s%N)
echo
echo "Took $(( (T1 - T0) / 1000 ))us"
umount $MNT
Before this change:
Took 28721us
After this change:
Took 5453us
That's about 5.3x times faster.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Inspired by the previous crash exposed by generic/795, we want to make
sure every folio from btrfs page cache is properly aligned to block
size.
This is especially important for bs > ps support, as every btrfs
infrastructure, e.g. extent map and extent state, requires strong block
alignment checks.
Furthermore, also output the minimal folio order from the inode mapping,
which is the determining factor during debugging, helping a lot pinning
down the final cause.
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
extent_buffer readahead should not be able to painfully stall a
search_slot and hog tree locks by getting stuck in direct reclaim.
If the allocation fails, that is fine, we simply fail to do the
readahead in that case.
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Boris Burkov <boris@bur.io>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Now that we have the btrfs_eb_prealloc struct to carry the allocation
and the "needs prealloc" signal, wire that up between the various
search_slot style callers down into alloc_extent_buffer.
If the prealloc struct indicates that it supports a nowait try, then
alloc_extent_buffer tries to allocate NOWAIT. If that succeeds, great.
Otherwise, we return EAGAIN and signal via the struct that preallocation
is required. The caller then does the allocation and tries again with
the eb, bfs, and folios wired through in the prealloc struct.
If unlock-and-allocate retries are not supported then we just use the
normal gfp flags like before.
Note that there are still two GFP_NOFS allocations, as far as I know,
that happen under the lock and cannot be preallocated:
- the __xa_cmpxchg to insert the eb into the eb xarray
- the xarray allocations for filemap_add_folio to add the folios to
the btree_inode mapping.
The former we could wire up with xa_reserve if we signaled the "prealloc
start" back up to the retry point. However, since there is no concept of
reservation in the filemap xarray, it seemed relatively unhelpful to
bother. These allocations are relatively small cached slab allocations,
so hopefully we can move the needle on reclaim stalls without reserving
them.
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Boris Burkov <boris@bur.io>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
In further preparation for supporting NOFAIL allocations with retries
outside the critical section, add a struct to carry the extent_buffer
and btrfs_folio_state we need to allocate.
Refactor the allocation pathways to use the new struct but with no
functional change. Wire empty prealloc structs in from callers.
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Boris Burkov <boris@bur.io>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
In preparation for preallocating extent_buffer data, factor eb
initialization away from specifically allocating it. This allows us to
allocate the eb, bfs, folios, etc. together in the main search_slot code
paths, but still share initialization code with the dummy/test/clone
allocation paths.
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Boris Burkov <boris@bur.io>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
In that function, we round down the start position and round up the
ending position.
But during the calculation of @len, we use "round_up(start + len,
sectorsize)", which is the rounded up end position, not the rounded up
length.
Which results a much larger length, and later we are still using
"start + len", which is completely incorrect.
Fix it by declaring a local @aligned_start and @aligned_len and use them
instead.
Fixes: bc42bda22345 ("btrfs: qgroup: Fix qgroup reserved space underflow by only freeing reserved ranges")
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Extent maps have the extra validation since commit 3f255ece2f1e ("btrfs:
introduce extra sanity checks for extent maps"), but extent states do
not have a similar check.
Introduce a basic alignment check for the following call sites, so that
we can cover all extent states inserted into the tree:
- insert_state_fast()
- insert_state()
- split_state()
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
Linus Torvalds
Anand Jain
Qu Wenruo
Filipe Manana
Guanghui Yang
Daniel Linjama
Hongling Zeng
Johannes Thumshirn
FAN YE
Avi Weiss
Leo Martins
ZhengYuan Huang
Shuangpeng Bai
James C. Owens
Sam Ho
Boris Burkov