Development of userspace BTRFS tools
Find a file
Qu Wenruo beb06b63cc btrfs-progs: lowmem check: Fix regression which screws up extent allocator
[BUG]
Commit 723427d7e6 ("btrfs-progs: check: change the way lowmem mode
traverses metadata") introduces a regression which could make some fsck
self test case to fail.

For fsck test case 004-no-dir-item, btrfs check --mode=lowmem --repair
can cause BUG_ON() with ret = -17 (-EEXIST) when committing transaction.

The problem happens with the following backtrace:

./btrfs(+0x22045)[0x555d0dade045]
./btrfs(+0x2216f)[0x555d0dade16f]
./btrfs(+0x29df1)[0x555d0dae5df1]
./btrfs(+0x2a142)[0x555d0dae6142]
./btrfs(btrfs_alloc_free_block+0x78)[0x555d0dae6202]
./btrfs(__btrfs_cow_block+0x177)[0x555d0dad00a2]
./btrfs(btrfs_cow_block+0x116)[0x555d0dad05a8]
./btrfs(commit_tree_roots+0x91)[0x555d0db1fd4f]
./btrfs(btrfs_commit_transaction+0x18c)[0x555d0db20100]
./btrfs(btrfs_fix_super_size+0x190)[0x555d0db005a4]
./btrfs(btrfs_fix_device_and_super_size+0x177)[0x555d0db00771]
./btrfs(cmd_check+0x1757)[0x555d0db4f6ab]
./btrfs(main+0x138)[0x555d0dace5dd]
/usr/lib/libc.so.6(__libc_start_main+0xea)[0x7fa5e4613f6a]
./btrfs(_start+0x2a)[0x555d0dacddda]

The bug is triggered by that, extent allocator considers range
[29360128, 29376512) as free and allocates it.  However when inserting
EXTENT_ITEM, btrfs finds there is already one tree block (fs tree root),
returning -EEXIST and causing the later BUG_ON().

[CAUSE]
The cause is in repair mode, lowmem check always pins all metadata
blocks.  However pinned metadata blocks will be unpined when transaction
commits, and will be marked as *FREE* space.

So later extent allocator will consider such range free and allocates
them incorrectly.

[FIX]
Don't pin metadata blocks without valid reason or preparation (like
discard all free space cache to re-calculate free space on next write).

Fixes: 723427d7e6 ("btrfs-progs: check: change the way lowmem mode traverses metadata")
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-01-03 17:09:07 +01:00
convert btrfs-progs: convert: add missing types header 2017-11-14 15:59:01 +01:00
Documentation btrfs-progs: docs: move the rescue fix-device-size command and update 2017-11-14 16:20:14 +01:00
image btrfs-progs: image: move sanitization to new file 2017-11-14 15:59:00 +01:00
kernel-lib btrfs-progs: kernel-lib: add generated tables.c 2017-08-24 19:06:31 +02:00
kernel-shared btrfs-progs: shared: cleanup includes in ulist.c 2017-03-08 13:00:47 +01:00
m4 btrfs-progs: build: add m4 macros for AC_CHECK_DEFINE 2016-06-24 14:05:44 +02:00
mkfs btrfs-progs: mkfs: refactor test_minimum_size to use the calculated minimal size 2017-11-14 15:59:00 +01:00
tests btrfs-progs: test: Add test image for lowmem mode referencer count mismatch false alert 2017-11-14 15:59:01 +01:00
travis btrfs-progs: build: move travis helper script to own directory 2017-10-13 20:15:50 +02:00
.gitignore btrfs-progs: kernel-lib: add generated tables.c 2017-08-24 19:06:31 +02:00
.travis.yml btrfs-progs: tests: fix path for travis helper script 2018-01-03 17:08:48 +01:00
64-btrfs-dm.rules btrfs-progs: udev: add rules for dm devices 2016-06-01 14:56:56 +02:00
Android.mk btrfs-progs: build: sync Makefile to Android.mk 2017-09-25 15:17:09 +02:00
androidcompat.h btrfs-progs: add compat header for android 2015-09-01 14:02:49 +02:00
autogen.sh btrfs-progs: build: let autotools include m4 2016-06-24 14:05:44 +02:00
backref.c btrfs-progs: backref: use separate list for indirect refs 2017-10-06 13:41:10 +02:00
backref.h btrfs-progs: move ulist.[ch] to kernel-shared 2017-03-08 13:00:47 +01:00
bcp Btrfs-progs: Correct path munging in bcp 2011-10-25 09:18:58 -04:00
btrfs-calc-size.c btrfs-progs: Removed missing header file, fixes compilation error 2017-09-25 15:16:52 +02:00
btrfs-completion btrfs-progs: cleanup whitespaces of btrfs-completion 2017-09-25 15:17:19 +02:00
btrfs-corrupt-block.c btrfs-progs: add crude error handling when transaction start fails 2017-09-08 16:15:05 +02:00
btrfs-crc.c btrfs-progs: btrfs-crc: make argc check more strict 2016-06-03 14:49:43 +02:00
btrfs-debug-tree.c btrfs-progs: move command definitions to commands.h 2017-08-24 19:08:24 +02:00
btrfs-debugfs btrfs-progs: right-align number columns in btrfs-debugfs output 2017-10-06 13:41:31 +02:00
btrfs-find-root.c btrfs-progs: move help defines to own header 2017-03-08 13:00:45 +01:00
btrfs-fragments.c btrfs-progs: use symbolic tree name when searching 2016-11-09 13:47:29 +01:00
btrfs-list.c btrfs-progs: subvol list: don't print deleted subvol as TOPLEVEL 2017-10-13 20:16:03 +02:00
btrfs-list.h btrfs-progs: subvol show: print more details about toplevel subvolume 2016-11-09 13:47:29 +01:00
btrfs-map-logical.c btrfs-progs: updated and add missing function attributes to the definition 2017-09-25 15:18:32 +02:00
btrfs-select-super.c btrfs-progs: Refactor write_all_supers and its callers to use btrfs_fs_info 2017-07-12 17:53:28 +02:00
btrfs-show-super.c btrfs-progs: fix build of btrfs-show-super 2018-01-03 17:09:00 +01:00
btrfs-zero-log.c btrfs-progs: add crude error handling when transaction start fails 2017-09-08 16:15:05 +02:00
btrfs.c btrfs-progs: rework option parser to use getopt for global options 2017-03-31 19:40:57 +02:00
btrfsck.h btrfs-progs: chunk-recovery: Fix a float point error 2015-11-19 18:34:30 +01:00
btrfstune.c btrfs-progs: add crude error handling when transaction start fails 2017-09-08 16:15:05 +02:00
CHANGES btrfs-progs: update CHANGES for v4.14 2017-11-14 16:20:14 +01:00
chunk-recover.c btrfs-progs: add crude error handling when transaction start fails 2017-09-08 16:15:05 +02:00
cmds-balance.c btrfs-progs: balance: minor wording adjustment for full balance warning 2017-04-19 18:07:03 +02:00
cmds-check.c btrfs-progs: lowmem check: Fix regression which screws up extent allocator 2018-01-03 17:09:07 +01:00
cmds-device.c btrfs-progs: doc: add description of missing and example, of device remove 2017-11-14 15:59:00 +01:00
cmds-fi-du.c btrfs-progs: fi du: don't call lookup_path_rootid for BTRFS_EMPTY_SUBVOL_DIR_OBJECTID 2017-09-08 16:15:05 +02:00
cmds-fi-usage.c btrfs-progs: don't use __u8 for fsid buffers 2017-11-14 15:59:00 +01:00
cmds-fi-usage.h btrfs-progs: don't use __u8 for fsid buffers 2017-11-14 15:59:00 +01:00
cmds-filesystem.c btrfs-progs: don't use __u8 for fsid buffers 2017-11-14 15:59:00 +01:00
cmds-inspect-dump-super.c btrfs-progs: Add zstd support 2017-09-25 15:17:13 +02:00
cmds-inspect-dump-tree.c btrfs-progs: drop blocksize from read_tree_block 2017-09-08 16:15:05 +02:00
cmds-inspect-tree-stats.c btrfs-progs: drop blocksize from read_tree_block 2017-09-08 16:15:05 +02:00
cmds-inspect.c btrfs-progs: inspect rootid: Allow a file to be specified 2017-09-08 16:15:05 +02:00
cmds-property.c btrfs-progs: move get_fsid() to utils.c 2017-10-06 13:23:27 +02:00
cmds-qgroup.c btrfs-progs: move help defines to own header 2017-03-08 13:00:45 +01:00
cmds-quota.c btrfs-progs: move help defines to own header 2017-03-08 13:00:45 +01:00
cmds-receive.c btrfs-progs: recieve: add missing short option E to getopt 2017-08-24 19:14:19 +02:00
cmds-replace.c btrfs-progs: move more mkfs declarations to the common header 2017-03-08 13:00:46 +01:00
cmds-rescue.c btrfs-progs: rescue: Introduce fix-device-size 2017-11-14 15:59:00 +01:00
cmds-restore.c btrfs-progs: Add zstd support 2017-09-25 15:17:13 +02:00
cmds-scrub.c btrfs-progs: move help defines to own header 2017-03-08 13:00:45 +01:00
cmds-send.c btrfs-progs: move help defines to own header 2017-03-08 13:00:45 +01:00
cmds-subvolume.c btrfs-progs: subvol: change set-default to also accept path 2017-11-14 15:59:00 +01:00
commands.h btrfs-progs: move command definitions to commands.h 2017-08-24 19:08:24 +02:00
common-defs.h btrfs-progs: move some common definitions to own header 2017-03-08 13:00:45 +01:00
configure.ac btrfs-progs: build: require libzstd support by default 2017-11-14 15:59:01 +01:00
COPYING fix COPYING 2007-06-12 15:41:08 -04:00
ctree.c btrfs-progs: drop blocksize argument from readahead_tree_block 2017-09-08 16:15:05 +02:00
ctree.h btrfs-progs: dump-tree: print c/o/s/r time of ROOT_ITEM 2018-01-03 17:08:57 +01:00
dir-item.c btrfs-progs: check: adjustments for further repair 2017-10-16 20:33:00 +02:00
dir-test.c btrfs-progs: add crude error handling when transaction start fails 2017-09-08 16:15:05 +02:00
disk-io.c btrfs-progs: handle transaction start failure in close_ctree 2017-09-08 16:15:05 +02:00
disk-io.h btrfs-progs: move transaction code out of disk-io 2017-09-08 16:15:05 +02:00
extent-cache.c btrfs-progs: extent-tree: Add add_merge_cache_extent function 2016-01-12 15:01:03 +01:00
extent-cache.h btrfs-progs: typo review of strings and comments 2016-06-01 14:56:56 +02:00
extent-tree.c btrfs-progs: mkfs: avoid BUG_ON for chunk allocation when ENOSPC happens 2017-11-14 15:59:00 +01:00
extent_io.c btrfs-progs: extent-cache: actually cache extent buffers 2017-10-06 13:41:06 +02:00
extent_io.h btrfs-progs: extent-cache: actually cache extent buffers 2017-10-06 13:41:06 +02:00
file-item.c btrfs-progs: convert: add support for converting reiserfs 2017-09-08 16:15:05 +02:00
file.c btrfs-progs: Refactor read_extent_data to use btrfs_fs_info 2017-07-12 17:53:13 +02:00
find-root.c btrfs-progs: drop blocksize from read_tree_block 2017-09-08 16:15:05 +02:00
find-root.h btrfs-progs: find-root: Allow btrfs-find-root to search chunk root even chunk root is corrupted 2016-02-26 17:27:58 +01:00
free-space-cache.c btrfs-progs: free-space-cache: fix endianity when reading from disk_key 2017-09-25 15:18:27 +02:00
free-space-cache.h btrfs-progs: check: add support to clear v1 free space cache 2016-11-09 13:12:47 +01:00
free-space-tree.c btrfs-progs: Refactor sectorsize users in free-space-tree.c 2017-07-03 13:35:10 +02:00
free-space-tree.h btrfs-progs: add btrfs_clear_free_space_tree() from the kernel 2016-11-23 11:07:05 +01:00
fsck.btrfs btrfs-progs: fsck.btrfs: Fix bashism and bad getopts processing 2015-05-21 16:53:44 +02:00
fsfeatures.c btrfs-progs: move fs features implementation to own file 2017-03-08 13:00:45 +01:00
fsfeatures.h btrfs-progs: Add zstd support 2017-09-25 15:17:13 +02:00
hash.h btrfs-progs: move btrfs_extref_hash() to hash.h 2016-10-25 14:28:36 +02:00
hasher.c Unaligned access fixes 2008-02-15 11:19:58 -05:00
help.c btrfs-progs: help: print multiple syntax schemas on separate lines 2017-11-14 15:59:00 +01:00
help.h btrfs-progs: updated and add missing function attributes to the definition 2017-09-25 15:18:32 +02:00
inode-item.c btrfs-progs: remove unused argument from btrfs_lookup_inode_ref 2017-03-08 13:00:47 +01:00
inode-map.c btrfs-progs: handle errors from btrfs_alloc_path 2016-09-21 14:12:38 +02:00
inode.c btrfs-progs: check: adjustments for further repair 2017-10-16 20:33:00 +02:00
INSTALL btrfs-progs: build: mention library dependency for reiserfs 2017-11-14 15:59:01 +01:00
internal.h btrfs-progs: libbtrfs: remove max/min macros from API 2016-03-14 13:42:47 +01:00
ioctl-test.c btrfs-progs: tests: make the ioctl-test actually useful 2016-10-24 14:57:19 +02:00
ioctl.h btrfs-progs: ioctl: fix compilation if ioctl.h is included standalone 2016-11-23 10:49:33 +01:00
kerncompat.h btrfs-progs: kerncompat: Fix re-definition of __bitwise 2017-03-16 17:40:35 +01:00
library-test.c btrfs-progs: library-test: add all exported headers 2017-03-31 14:17:48 +02:00
Makefile btrfs-progs: image: move sanitization to new file 2017-11-14 15:59:00 +01:00
Makefile.extrawarn btrfs-progs: build: allow to build with various compiler warnings 2015-10-21 14:29:26 +02:00
Makefile.inc.in btrfs-progs: Add zstd support 2017-09-25 15:17:13 +02:00
messages.c btrfs-progs: add prefix to message helpers 2017-03-31 14:17:48 +02:00
messages.h btrfs-progs: fix debugging macro checks 2017-09-25 15:18:17 +02:00
print-tree.c btrfs-progs: dump-tree: print c/o/s/r time of ROOT_ITEM 2018-01-03 17:08:57 +01:00
print-tree.h btrfs-progs: print-tree: rename item callbacks to match the key name 2017-09-08 16:15:05 +02:00
props.c btrfs-progs: prop: also allow "none" to disable compression 2017-11-14 15:59:00 +01:00
props.h btrfs-progs: unify header file inclusion protections 2015-01-21 17:49:26 +01:00
qgroup-verify.c btrfs-progs: drop blocksize from read_tree_block 2017-09-08 16:15:05 +02:00
qgroup-verify.h btrfs-progs: check: write corrected qgroup info to disk 2016-07-13 18:44:26 +02:00
qgroup.c btrfs-progs: qgroup: split update_qgroup to reduce arguments 2017-11-14 15:59:01 +01:00
qgroup.h btrfs-progs: remove unused function btrfs_get_path_rootid 2016-11-09 13:47:28 +01:00
quick-test.c btrfs-progs: add crude error handling when transaction start fails 2017-09-08 16:15:05 +02:00
random-test.c btrfs-progs: add crude error handling when transaction start fails 2017-09-08 16:15:05 +02:00
rbtree-utils.c Btrfs-progs: break out rbtree util functions 2014-10-14 10:39:39 +02:00
rbtree-utils.h btrfs-progs: libbtrfs, make exported headers compatible with C++ 2014-11-03 23:50:50 +01:00
README.md btrfs-progs: README: update links, enhance contribution section 2017-09-25 15:17:04 +02:00
repair.c btrfs-progs: factor out repair mode 2016-07-04 15:22:53 +02:00
repair.h btrfs-progs: factor out repair mode 2016-07-04 15:22:53 +02:00
root-tree.c btrfs-progs: Use more strict check to read out tree root 2017-05-02 16:33:04 +02:00
send-dump.c btrfs-progs: send: always print a space after path in dump 2017-04-19 19:33:43 +02:00
send-dump.h btrfs-progs: introduce new send-dump object 2016-12-14 15:06:34 +01:00
send-stream.c btrfs-progs: fix btrfs send & receive with -e flag 2017-04-07 18:51:19 +02:00
send-stream.h btrfs-progs: add missing includes to header files 2015-06-10 02:52:21 +02:00
send-utils.c btrfs-progs: handle failed strdup in subvol_uuid_search2 2016-12-14 15:06:36 +01:00
send-utils.h btrfs-progs: fixup API after change in subvol_uuid_search 2016-12-14 15:06:36 +01:00
send.h btrfs-progs: Introduce kernel sizes to cleanup large intermediate number 2017-03-08 13:00:45 +01:00
show-blocks btrfs-progs: typo review of strings and comments 2016-06-01 14:56:56 +02:00
string-table.c btrfs-progs: constify string arguments where appropriate 2016-10-03 15:07:23 +02:00
string-table.h btrfs-progs: constify string arguments where appropriate 2016-10-03 15:07:23 +02:00
super-recover.c btrfs-progs: Refactor write_all_supers and its callers to use btrfs_fs_info 2017-07-12 17:53:28 +02:00
task-utils.c btrfs-progs: Check periodic.timer_fd's value before use 2015-11-16 14:23:45 +01:00
task-utils.h btrfs-progs: unify header file inclusion protections 2015-01-21 17:49:26 +01:00
transaction.c btrfs-progs: don't start or commit after transaction abort 2017-09-08 16:15:05 +02:00
transaction.h btrfs-progs: start framework for transaction abort 2017-09-08 16:15:05 +02:00
utils-lib.c btrfs-progs: move lookup_path_rootid to library utils 2016-11-24 13:06:36 +01:00
utils.c btrfs-progs: extent-cache: actually cache extent buffers 2017-10-06 13:41:06 +02:00
utils.h btrfs-progs: extent-cache: actually cache extent buffers 2017-10-06 13:41:06 +02:00
uuid-tree.c btrfs-progs: use ioctl search headers everywhere 2016-05-11 15:53:16 +02:00
version.h.in btrfs-progs: re-introduce BTRFS_BUILD_VERSION in version.h 2015-03-16 18:40:06 +01:00
version.sh Btrfs progs v4.14 2017-11-20 17:39:52 +01:00
volumes.c btrfs-progs: rescue: Introduce fix-device-size 2017-11-14 15:59:00 +01:00
volumes.h btrfs-progs: rescue: Introduce fix-device-size 2017-11-14 15:59:00 +01:00

Btrfs-progs build status coverity status

Userspace utilities to manage btrfs filesystems. License: GPLv2.

Btrfs is a copy on write (COW) filesystem for Linux aimed at implementing advanced features while focusing on fault tolerance, repair and easy administration.

This repository hosts following utilities:

  • btrfs — the main administration tool (manual page)
  • mkfs.btrfs — utility to create the filesystem (manual page)

See INSTALL for build instructions and tests/README.md for testing information.

Release cycle

The major version releases are time-based and follow the cycle of the linux kernel releases. The cycle usually takes 2 months. A minor version releases may happen in the meantime if there are bug fixes or minor useful improvements queued.

The release tags are signed with a GPG key ID F2B4 1200 C54E FB30 380C 1756 C565 D5F9 D76D 583B, release tarballs are hosted at kernel.org. See file CHANGES or changelogs on wiki.

Reporting bugs

There are several ways, each has its own specifics and audience that can give feedback or work on a fix. The following list is sorted in the order of preference:

  • github issue tracker
  • to the mailing list linux-btrfs@vger.kernel.org -- (not required to subscribe), beware that the mail might get overlooked in other traffic
  • IRC (irc.freenode.net #btrfs) -- good for discussions eg. if a bug is already known, but reports could miss developers' attention
  • bugzilla.kernel.org -- (requires registration), set the product to Filesystems and component Btrfs, please put 'btrfs-progs' into the subject so it's clear that it's not a kernel bug report

Development

The patch submissions, development or general discussions take place at linux-btrfs@vger.kernel.org mailinglist, subsciption is not required to post.

The GitHub pull requests will not be accepted directly, the preferred way is to send patches to the mailinglist instead. You can link to a branch in any git repository if the mails do not make it to the mailinglist or just for convenience (makes it easier to test).

The development model of btrfs-progs shares a lot with the kernel model. The github way is different in some ways. We, the upstream community, expect that the patches meet some criteria (often lacking in github contributions):

  • one logical change per patch: eg. not mixing bugfixes, cleanups, features etc., sometimes it's not clear and will be usually pointed out during reviews
  • proper subject line: eg. prefix with btrfs-progs: subpart, ... , descriptive yet not too long, see git log --oneline for some inspiration
  • proper changelog: the changelogs are often missing or lacking explanation why the change was made, or how is something broken, what are user-visible effects of the bug or the fix, how does an improvement help or the intended usecase
  • the Signed-off-by line: this documents who authored the change, you can read more about the The Developer's Certificate of Origin (chapter 11)
    • if you are not used to the signed-off style, your contributions won't be rejected just because of it's missing, the Author: tag will be added as a substitute in order to allow contributions without much bothering with formalities

Documentation updates

Documentation fixes or updates do not need much explanation so sticking to the code rules in the previous section is not necessary. Github pull requests are OK, patches could be sent to me directly and not required to be also in the mailinglist. Pointing out typos via IRC also works, although might get accidentally lost in the noise.

References