Messages in this thread | | | From | David Sterba <> | Subject | [GIT PULL] Btrfs updates for 6.5 | Date | Mon, 26 Jun 2023 20:14:42 +0200 |
| |
Hi,
there are mainly core changes, refactoring and optimizations. Performance is improved in some areas, overall there may be a cumulative improvement due to refactoring that removed lookups in the IO path or simplified IO submission tracking.
No merge conflicts. Please pull, thanks.
Core:
- submit IO synchronously for fast checksums (crc32c and xxhash), remove high priority worker kthread
- read extent buffer in one go, simplify IO tracking, bio submission and locking
- remove additional tracking of redirtied extent buffers, originally added for zoned mode but actually not needed
- track ordered extent pointer in bio to avoid rbtree lookups during IO
- scrub, use recovered data stripes as cache to avoid unnecessary read
- in zoned mode, optimize logical to physical mappings of extents
- remove PageError handling, not set by VFS nor writeback
- cleanups, refactoring, better structure packing
- lots of error handling improvements
- more assertions, lockdep annotations
- print assertion failure with the exact line where it happens
- tracepoint updates
- more debugging prints
Performance:
- speedup in fsync(), better tracking of inode logged status can avoid transaction commit
- IO path structures track logical offsets in data structures and does not need to look it up
User visible changes:
- don't commit transaction for every created subvolume, this can reduce time when many subvolumes are created in a batch
- print affected files when relocation fails
- trigger orphan file cleanup during START_SYNC ioctl
Notable fixes:
- fix crash when disabling quota and relocation
- fix crashes when removing roots from drity list
- fix transacion abort during relocation when converting from newer profiles not covered by fallback
- in zoned mode, stop reclaiming block groups if filesystem becomes read-only
- fix rare race condition in tree mod log rewind that can miss some btree node slots
- with enabled fsverity, drop up-to-date page bit in case the verification fails
---------------------------------------------------------------- The following changes since commit 45a3e24f65e90a047bef86f927ebdc4c710edaa1:
Linux 6.4-rc7 (2023-06-18 14:06:27 -0700)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-6.5
for you to fetch changes up to 8a4a0b2a3eaf75ca8854f856ef29690c12b2f531:
btrfs: fix race between quota disable and relocation (2023-06-19 20:29:25 +0200)
---------------------------------------------------------------- Anand Jain (11): btrfs: use SECTOR_SHIFT to convert physical offset to LBA btrfs: use SECTOR_SHIFT to convert LBA to physical offset btrfs: reduce struct btrfs_fs_devices size by moving fsid_change btrfs: streamline fsid checks in alloc_fs_devices btrfs: merge calls to alloc_fs_devices in device_list_add btrfs: add comment about metadata_uuid in btrfs_fs_devices btrfs: return bool from check_tree_block_fsid instead of int btrfs: simplify fsid and metadata_uuid comparisons btrfs: simplify how changed fsid and metadata_uuid is checked btrfs: consolidate uuid comparisons in btrfs_validate_super btrfs: add and fix comments in btrfs_fs_devices
Ben Dooks (1): btrfs: make btrfs_compressed_bioset static
Boris Burkov (2): btrfs: warn on invalid slot in tree mod log rewind btrfs: insert tree mod log move in push_node_left
Christoph Hellwig (80): btrfs: submit IO synchronously for fast checksum implementations btrfs: determine synchronous writers from bio or writeback control btrfs: remove hipri_workers workqueue btrfs: fix dirty_metadata_bytes for redirtied buffers btrfs: don't hold an extra reference for redirtied buffers btrfs: mark extent_buffer_under_io static btrfs: subpage: fix error handling in end_bio_subpage_eb_writepage btrfs: move setting the buffer uptodate out of validate_extent_buffer btrfs: merge verify_parent_transid and btrfs_buffer_uptodate btrfs: always read the entire extent_buffer btrfs: don't use btrfs_bio_ctrl for extent buffer reading btrfs: remove the mirror_num argument to btrfs_submit_compressed_read btrfs: use a separate end_io handler for read_extent_buffer btrfs: do not try to unlock the extent for non-subpage metadata reads btrfs: return bool from lock_extent_buffer_for_io btrfs: submit a writeback bio per extent_buffer btrfs: move page locking from lock_extent_buffer_for_io to write_one_eb btrfs: don't use btrfs_bio_ctrl for extent buffer writing btrfs: use a separate end_io handler for extent_buffer writing btrfs: remove the extent_buffer lookup in btree block checksumming btrfs: remove the io_pages field in struct extent_buffer btrfs: stop using PageError for extent_buffers btrfs: don't check for uptodate pages in read_extent_buffer_pages btrfs: stop using lock_extent in btrfs_buffer_uptodate btrfs: use per-buffer locking for extent_buffer reading btrfs: merge write_one_subpage_eb into write_one_eb btrfs: fix the btrfs_get_global_root return value btrfs: convert btrfs_get_global_root to use a switch statement btrfs: remove a pointless NULL check in btrfs_lookup_fs_root btrfs: optimize out btrfs_is_zoned for !CONFIG_BLK_DEV_ZONED btrfs: don't call btrfs_record_physical_zoned for failed append btrfs: mark the len field in struct btrfs_ordered_sum as unsigned btrfs: rename the bytenr field in struct btrfs_ordered_sum to logical btrfs: optimize the logical to physical mapping for zoned writes btrfs: record orig_physical only for the original bio btrfs: move split_extent_map to extent_map.c btrfs: reorder conditions in btrfs_extract_ordered_extent btrfs: return the new ordered_extent from btrfs_split_ordered_extent btrfs: split btrfs_alloc_ordered_extent to allocation and insertion helpers btrfs: atomically insert the new extent in btrfs_split_ordered_extent btrfs: handle completed ordered extents in btrfs_split_ordered_extent btrfs: defer splitting of ordered extents until I/O completion btrfs: pass the new logical address to split_extent_map btrfs: remove unused BTRFS_MAP_DISCARD btrfs: optimize simple reads in btrfsic_map_block btrfs: remove unused btrfs_map_block btrfs: rename __btrfs_map_block to btrfs_map_block btrfs: open code btrfs_map_sblock btrfs: open code need_full_stripe conditions btrfs: fix range_end calculation in extent_write_locked_range btrfs: factor out a btrfs_verify_page helper btrfs: fix fsverify read error handling in end_page_read btrfs: don't check PageError in btrfs_verify_page btrfs: don't fail writeback when allocating the compression context fails btrfs: rename cow_file_range_async to run_delalloc_compressed btrfs: don't check PageError in __extent_writepage btrfs: stop setting PageError in the data I/O path btrfs: remove PAGE_SET_ERROR btrfs: remove non-standard extent handling in __extent_writepage_io btrfs: move writeback_control::nr_to_write update to __extent_writepage btrfs: only call __extent_writepage_io from extent_write_locked_range btrfs: don't treat zoned writeback as being from an async helper thread btrfs: fix file_offset for REQ_BTRFS_ONE_ORDERED bios that get split btrfs: limit write bios to a single ordered extent btrfs: merge the two calls to btrfs_add_ordered_extent in run_delalloc_nocow btrfs: pass an ordered_extent to btrfs_reloc_clone_csums btrfs: pass an ordered_extent to btrfs_submit_compressed_write btrfs: remove btrfs_add_ordered_extent btrfs: add a is_data_bbio helper btrfs: open code btrfs_bio_end_io in btrfs_dio_submit_io btrfs: add an ordered_extent pointer to struct btrfs_bio btrfs: use bbio->ordered in btrfs_csum_one_bio btrfs: factor out a can_finish_ordered_extent helper btrfs: factor out a btrfs_queue_ordered_fn helper btrfs: add a btrfs_finish_ordered_extent helper btrfs: open code end_extent_writepage in end_bio_extent_writepage btrfs: use btrfs_finish_ordered_extent to complete compressed writes btrfs: use btrfs_finish_ordered_extent to complete direct writes btrfs: use btrfs_finish_ordered_extent to complete buffered writes btrfs: set FMODE_CAN_ODIRECT instead of a dummy direct_IO method
David Sterba (13): btrfs: open code set_extent_defrag btrfs: open code set_extent_delalloc btrfs: open code set_extent_new btrfs: open code set_extent_dirty btrfs: open code set_extent_bits_nowait btrfs: open code set_extent_bits btrfs: drop NOFAIL from set_extent_bit allocation masks btrfs: pass NOWAIT for set/clear extent bits as another bit btrfs: drop gfp from parameter extent state helpers btrfs: print assertion failure report and stack trace from the same line btrfs: add xxhash to fast checksum implementations btrfs: disable allocation warnings for compression workspaces btrfs: add block-group tree to lockdep classes
Filipe Manana (51): btrfs: unexport btrfs_prev_leaf() btrfs: make btrfs_free_device() static btrfs: tag as unlikely the key comparison when checking sibling keys btrfs: avoid extra memory allocation when copying free space cache btrfs: avoid searching twice for previous node when merging free space entries btrfs: use precomputed end offsets at do_trimming() btrfs: simplify arguments to tree_insert_offset() btrfs: assert proper locks are held at tree_insert_offset() btrfs: assert tree lock is held when searching for free space entries btrfs: assert tree lock is held when linking free space btrfs: assert tree lock is held when removing free space entries btrfs: use inode_logged() at need_log_inode() btrfs: use inode_logged() at btrfs_record_unlink_dir() btrfs: update comments at btrfs_record_unlink_dir() to be more clear btrfs: remove pointless label and goto at btrfs_record_unlink_dir() btrfs: change for_rename argument of btrfs_record_unlink_dir() to bool btrfs: fix comment referring to no longer existing btrfs_clean_tree_block() btrfs: reorder some members of struct btrfs_delayed_ref_head btrfs: remove unused is_head field from struct btrfs_delayed_ref_node btrfs: remove pointless in_tree field from struct btrfs_delayed_ref_node btrfs: use a bool to track qgroup record insertion when adding ref head btrfs: make insert_delayed_ref() return a bool instead of an int btrfs: get rid of label and goto at insert_delayed_ref() btrfs: assert correct lock is held at btrfs_select_ref_head() btrfs: use bool type for delayed ref head fields that are used as booleans btrfs: use a single switch statement when initializing delayed ref head btrfs: remove unnecessary prototype declarations at disk-io.c btrfs: make btrfs_destroy_delayed_refs() return void btrfs: update documentation for a block group's bg_list member btrfs: add missing error handling when logging operation while COWing extent buffer btrfs: fix extent buffer leak after tree mod log failure at split_node() btrfs: avoid tree mod log ENOMEM failures when we don't need to log btrfs: do not BUG_ON() on tree mod log failure at __btrfs_cow_block() btrfs: do not BUG_ON() on tree mod log failure at balance_level() btrfs: rename enospc label to out at balance_level() btrfs: avoid unnecessarily setting the fs to RO and error state at balance_level() btrfs: abort transaction at balance_level() when left child is missing btrfs: abort transaction at update_ref_for_cow() when ref count is zero btrfs: do not BUG_ON() on tree mod log failures at push_nodes_for_insert() btrfs: do not BUG_ON() on tree mod log failure at insert_new_root() btrfs: do not BUG_ON() on tree mod log failures at insert_ptr() btrfs: do not BUG_ON() on tree mod log failures at btrfs_del_ptr() btrfs: replace BUG_ON() at split_item() with proper error handling btrfs: do not BUG_ON() when dropping inode items from log root btrfs: send: do not BUG_ON() on unexpected symlink data extent btrfs: do not BUG_ON on failure to get dir index for new snapshot btrfs: do not BUG_ON after failure to migrate space during truncation btrfs: fix race when deleting quota root from the dirty cow roots list btrfs: fix race when deleting free space root from the dirty cow roots list btrfs: add comment to struct btrfs_fs_info::dirty_cowonly_roots btrfs: fix race between quota disable and relocation
Jeff Layton (1): btrfs: update i_version in update_dev_time
Jiapeng Chong (1): btrfs: scrub: remove more unused functions
Johannes Thumshirn (1): btrfs: unexport btrfs_run_discard_work and make it static
Josef Bacik (12): btrfs: move btrfs_check_trunc_cache_free_space into block-rsv.c btrfs: remove level argument from btrfs_set_block_flags btrfs: simplify btrfs_check_leaf_* helpers into a single helper btrfs: add btrfs_tree_block_status definitions to tree-checker.h btrfs: use btrfs_tree_block_status for leaf item errors btrfs: extend btrfs_leaf_check to return btrfs_tree_block_status btrfs: add __btrfs_check_node helper btrfs: move btrfs_verify_level_key into tree-checker.c btrfs: move split_flags/combine_flags helpers to inode-item.h btrfs: add __KERNEL__ check for btrfs_no_printk btrfs: add a btrfs_csum_type_size helper btrfs: rename del_ptr to btrfs_del_ptr and export it
Matt Corallo (1): btrfs: add handling for RAID1C23/DUP to btrfs_reduce_alloc_profile
Naohiro Aota (6): btrfs: export bitmap_test_range_all_{set,zero} btrfs: delete unused BGs while reclaiming BGs btrfs: move out now unused BG from the reclaim list btrfs: bail out reclaim process if filesystem is read-only btrfs: reinsert BGs failed to reclaim btrfs: tracepoints: also show actual number of the outstanding extents
Qu Wenruo (11): btrfs: print-tree: pass const extent buffer pointer btrfs: improve leaf dump and error handling btrfs: output affected files when relocation fails btrfs: scrub: use recovered data stripes as cache to avoid unnecessary read btrfs: handle tree backref walk error properly btrfs: trigger orphan inode cleanup during START_SYNC ioctl btrfs: subpage: dump extra subpage bitmaps for debug btrfs: subpage: make alloc_extent_buffer() handle previously uptodate range efficiently btrfs: use the same uptodate variable for end_bio_extent_readpage() btrfs: scrub: remove scrub_ctx::csum_list member btrfs: scrub: remove btrfs_fs_info::scrub_wr_completion_workers
Sweet Tea Dorminy (1): btrfs: don't commit transaction for every subvol create
Tejun Heo (1): btrfs: use alloc_ordered_workqueue() to create ordered workqueues
Tom Rix (1): btrfs: simplify transid initialization in btrfs_ioctl_wait_sync
fs/btrfs/async-thread.c | 44 +- fs/btrfs/async-thread.h | 3 + fs/btrfs/bio.c | 122 +++--- fs/btrfs/bio.h | 29 +- fs/btrfs/block-group.c | 47 ++- fs/btrfs/block-group.h | 9 +- fs/btrfs/block-rsv.c | 19 + fs/btrfs/block-rsv.h | 2 + fs/btrfs/btrfs_inode.h | 23 +- fs/btrfs/check-integrity.c | 21 +- fs/btrfs/compression.c | 50 +-- fs/btrfs/compression.h | 7 +- fs/btrfs/ctree.c | 429 +++++++++++++------- fs/btrfs/ctree.h | 6 +- fs/btrfs/defrag.c | 3 +- fs/btrfs/delayed-ref.c | 110 ++--- fs/btrfs/delayed-ref.h | 25 +- fs/btrfs/dev-replace.c | 6 +- fs/btrfs/discard.c | 34 +- fs/btrfs/discard.h | 1 - fs/btrfs/disk-io.c | 451 ++++----------------- fs/btrfs/disk-io.h | 7 +- fs/btrfs/extent-io-tree.c | 37 +- fs/btrfs/extent-io-tree.h | 62 +-- fs/btrfs/extent-tree.c | 186 ++++----- fs/btrfs/extent-tree.h | 2 +- fs/btrfs/extent_io.c | 848 +++++++++++---------------------------- fs/btrfs/extent_io.h | 11 +- fs/btrfs/extent_map.c | 110 ++++- fs/btrfs/extent_map.h | 6 +- fs/btrfs/file-item.c | 90 ++--- fs/btrfs/file-item.h | 1 + fs/btrfs/file.c | 12 +- fs/btrfs/free-space-cache.c | 122 +++--- fs/btrfs/free-space-cache.h | 2 - fs/btrfs/free-space-tree.c | 3 + fs/btrfs/fs.h | 5 +- fs/btrfs/inode-item.h | 16 + fs/btrfs/inode.c | 592 ++++++++++++++++----------- fs/btrfs/ioctl.c | 23 +- fs/btrfs/locking.c | 5 +- fs/btrfs/lzo.c | 6 +- fs/btrfs/messages.c | 8 - fs/btrfs/messages.h | 15 +- fs/btrfs/misc.h | 20 + fs/btrfs/ordered-data.c | 442 +++++++++++--------- fs/btrfs/ordered-data.h | 27 +- fs/btrfs/print-tree.c | 16 +- fs/btrfs/print-tree.h | 4 +- fs/btrfs/qgroup.c | 20 +- fs/btrfs/raid56.c | 49 ++- fs/btrfs/raid56.h | 3 + fs/btrfs/relocation.c | 61 +-- fs/btrfs/relocation.h | 3 +- fs/btrfs/scrub.c | 125 ++---- fs/btrfs/send.c | 16 +- fs/btrfs/subpage.c | 97 ++--- fs/btrfs/subpage.h | 12 +- fs/btrfs/super.c | 1 - fs/btrfs/tests/extent-io-tests.c | 16 +- fs/btrfs/transaction.c | 16 +- fs/btrfs/transaction.h | 3 - fs/btrfs/tree-checker.c | 158 ++++++-- fs/btrfs/tree-checker.h | 29 +- fs/btrfs/tree-log.c | 62 +-- fs/btrfs/tree-log.h | 2 +- fs/btrfs/tree-mod-log.c | 257 ++++++++++-- fs/btrfs/volumes.c | 173 ++++---- fs/btrfs/volumes.h | 57 ++- fs/btrfs/zlib.c | 2 +- fs/btrfs/zoned.c | 159 ++++---- fs/btrfs/zoned.h | 8 +- fs/btrfs/zstd.c | 2 +- include/trace/events/btrfs.h | 39 +- tools/objtool/check.c | 1 - 75 files changed, 2780 insertions(+), 2710 deletions(-)
| |