For extent tree search, we are only search two things: either
EXTENT_ITEM/METADATA_ITEM (inlined) or SHARED_BLOCK_REF/SHARED_DATA_REF
(keyed).
Except certain situation like cache_block_group(), we never read tree
blocks in a forward or backward sequence.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The whole maybe_repair_root_item() and repair_root_items() functions are
introduced to handle an ancient bug in v3.17.
However in certain extent tree corruption case, such early exit would
only exit the whole check process early on, preventing user to know
what's really wrong about the fs.
So this patch will allow the check to continue, since the ancient bug is
no long that common.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Enhance the clean-tests.sh script to detach all loop devices created by
tests, lookup is based on the path prefix. Now 'make test-clean' is
supposed to prepare the environment for next test round.
Signed-off-by: David Sterba <dsterba@suse.com>
Update the summary of 'fi usage' where the multiple profiles will be
listed by type, like:
Multiple profiles: yes (data, metadata)
The string is returned from btrfs_test_for_multiple_profiles so the
callers don't have to assemble it together from the other profile
strings.
Signed-off-by: David Sterba <dsterba@suse.com>
The warning header was printed always, even if there weren't multiple
block group profiles. This also fixes the mixed block group profile
detection.
Signed-off-by: David Sterba <dsterba@suse.com>
Use simpler output format for easier parsing and place each block group
type on a separate line.
Example output:
WARNING: Multiple block group profiles detected, see 'man btrfs(5)'.
WARNING: Data: single, raid1
Signed-off-by: David Sterba <dsterba@suse.com>
Move 'single' as the first in the list of the multiple block groups, as
it's the default block group and the simplest.
Example output:
WARNING: data -> [single, raid1], metadata -> [single], system -> [single]
WARNING: data+metadata -> [single], system -> [raid1]
Signed-off-by: David Sterba <dsterba@suse.com>
Simpify sprint_profiles so it does not take the output parameters
optionally and add stubs to btrfs_test_for_multiple_profiles_by_fd.
This allows to remove all conditionals and reduce parameters of
sprint_profiles so that the output is returned directly.
Signed-off-by: David Sterba <dsterba@suse.com>
The term 'mixed' is confusing as it's commonly used for mised block
group profiles created by 'mkfs.btrfs --mixed'. We're interested in
multiple profiles for each type, so use the term 'multiple'.
Signed-off-by: David Sterba <dsterba@suse.com>
Several commands now warn about multiple profiles as it is considered a
potentially problematic filesystem state that might need user attention.
- device delete
- device add
- balance pause
- balance cancel
- filesystem usage
WARNING: Multiple profiles detected. See 'man btrfs(5)'.
WARNING: data -> [raid1, single]
Signed-off-by: David Sterba <dsterba@suse.com>
We can easily get the level from @eb parameter, thus the level is not
needed.
This is inspired by the work of Marek in U-boot.
Cc: Marek Behun <marek.behun@nic.cz>
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Perhaps a rather trivial example but I find it helpful to see usage
examples. Maybe it will help others.
Pull-request: #244
Author: Alexandru Ungureanu <khakcarot@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This example in device replace doesn't render properly in browsers,
think it's just misuse of syntax. Unfortunately, the github preview
renderer doesn't seem to catch that. Otherwise, manual page and html
rendering is ok.
The syntax uses Example delimited block:
.Heading
====
contents
====
Pull-request: #245
Author: Alexandru Ungureanu <khakcarot@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[PROBLEM]
We want to inject $INSTRUMENT (mostly valgrind) before btrfs command but
after root_helper.
Currently we won't inject $INSTRUMENT at all if we are using
root_helper.
This means the coverage is not good enough.
[FIX]
This patch introduce a new function, expand_command(), to handle all
parameter/argument injection, including existing 'btrfs check' inject.
This function will:
- Detect where to inject $INSTRUMENT
If we have root_helper and the command is target command
(btrfs/mkfs.btrfs/btrfs-convert), then we inject $INSTRUMENT after
root_helper.
If we don't have root_helper, and the command is target command,
we inject $INSTRUMENT before the command.
Or we don't inject $INSTRUMENT (it's not the target command).
- Use existing spec facility to inject extra arguments
- Use an array to restore to result
To avoid bash interpret the IFS inside path/commands.
Now we can make sure no matter if we use root_helper, $INSTRUMENT is
always injected corrected.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Since run_check_stdout() can insert INSTRUMENT for all btrfs related
programs, which could easily pollute the stdout, any caller of
run_check_stdout() should do proper filter.
The following callers are affected:
- misc/004
Filter the output of "btrfs ins min-dev-size"
- misc/009
- misc/013
- misc/024
They are all calling "btrfs ins rootid", so introduce get_subvolid()
function to grab the subvolid properly.
- misc/031
Loose the filter for "btrfs qgroup show". No need for "tail -n 1".
So we still have the same coverage, but now these tests won't cause
false alert if we insert INSTRUMENT for all btrfs commands.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
For environments that require certified implementations of cryptographic
primitives allow to select a library providing them. The requirements
are SHA256 and BLAKE2 (with the 2b variant and 256 bit digest).
For now there are two: libgrcrypt and libsodium (openssl does not
provide the BLAKE2b-256). Accellerated versions are typically provided
and automatically selected.
Signed-off-by: David Sterba <dsterba@suse.com>
If we're compiling using system header, we should include
<btrfs/crc32c.h> other than "btrfs/crc32.h".
Fixes: 2efe160bc7 ("btrfs-progs: move name hashing functions to ctree.h and delete hash.h")
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
If we have memory allocation failure in add_cache_extent(), it will
simply exit with one error message.
That's definitely not proper, especially when all but one call sites
have handled the error.
This patch will return -ENOMEM for add_cache_extent(), and fix the only
call site which doesn't handle error from it.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This member can be fetched from eb::fs_info, and no caller really
depends on that member to determine if an eb is dummy. We have eb::flags
to determine that.
Kernel doesn't have such member either.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The code using it has been removed in 4e34bdb868 ("btrfs-progs: Remove
old commented code") and at that time it wasn't in use since
95d3f20b51 ("Mixed back reference (FORWARD ROLLING FORMAT CHANGE)").
Signed-off-by: David Sterba <dsterba@suse.com>
As the TEST_TOP is often only a base path, the slash gets doubled. This
is not a problem as the slashes are in the middle of the path and cannot
be confused with the initial "//" that gets interpreted as a network
path. Update all the test drivers.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
[ update changelog ]
Signed-off-by: David Sterba <dsterba@suse.com>
Commit 31f477ee ("btrfs-progs: mkfs-tests: skip test if truncate fails
with EFBIG") tried to detect a failure in truncate command by checking
the $? expecting it to be an errno, when it actually returns 0 or 1.
To fix this test just check if the command failed (returned 1) and look
for the output, skipping the test if the OS cannot create a 6E file.
Fixes: #241
Tested-by: Erhard Furtner <erhard_f@mailbox.org>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add the warning to 'device usage' and 'filesystem df'.
Signed-off-by: Goffredo Baroncelli <kreijack@inwid.it>
Signed-off-by: David Sterba <dsterba@suse.com>
A new line in the "Overall" section is added to inform that 'Multiple
profiles' are present.
Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Signed-off-by: David Sterba <dsterba@suse.com>
Add a check in some btrfs subcommands to detect if a filesystem
has mixed profiles for data/metadata/system. In this case
a warning is showed.
Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Signed-off-by: David Sterba <dsterba@suse.com>
Add code to show a warning if a mixed profiles filesystem is detected.
Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Signed-off-by: David Sterba <dsterba@suse.com>
Add BTRFS_EXTENDED_PROFILE_MASK to consider also the
BTRFS_AVAIL_ALLOC_BIT_SINGLE bit.
Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Signed-off-by: David Sterba <dsterba@suse.com>
- complete the function btrfs_err_str adding some missing cases
- sync the enum btrfs_err_code (in libbtrfsutil/btrfs.h) with the
rest of the codes (user space and kernel space).
- add missing fields to btrfs_raid_array[] for raid1c[34]
Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
btrfs check can return strange return value for shell:
[Inferior 1 (process 48641) exited with code 0213]
^^^^
[CAUSE]
It's caused by the incorrect handling of qgroup error.
qgroup_report_ret can be -117 (-EUCLEAN), using that value with exit()
can cause overflow, causing return value not properly recognized.
[FIX]
Fix it by sanitize the return value to 0 or 1.
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Reviewed-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: Adam Borowski <kilobyte@angband.pl>
Signed-off-by: David Sterba <dsterba@suse.com>
Some scripts can still rely on this message, so make it available with
-vv, so -v stays sane.
Fixes: #127
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
Valgrind reports memory leak for fsck/012, even after the image is
repair, the memory leak can still be reproduced.
==107060== HEAP SUMMARY:
==107060== in use at exit: 176 bytes in 1 blocks
==107060== total heap usage: 10,647 allocs, 10,646 frees, 3,000,654 bytes allocated
==107060==
==107060== 176 bytes in 1 blocks are definitely lost in loss record 1 of 1
==107060== at 0x483BB65: calloc (vg_replace_malloc.c:762)
==107060== by 0x1BD953: read_one_block_group (extent-tree.c:2661)
==107060== by 0x1BDBD8: btrfs_read_block_groups (extent-tree.c:2719)
==107060== by 0x1B3A2C: btrfs_setup_all_roots (disk-io.c:1024)
==107060== by 0x1B44CA: __open_ctree_fd (disk-io.c:1299)
==107060== by 0x1B46C6: open_ctree_fs_info (disk-io.c:1345)
==107060== by 0x16952E: cmd_check (main.c:10154)
==107060== by 0x11CDC6: cmd_execute (commands.h:125)
==107060== by 0x11D712: main (btrfs.c:386)
==107060==
==107060== LEAK SUMMARY:
==107060== definitely lost: 176 bytes in 1 blocks
==107060== indirectly lost: 0 bytes in 0 blocks
==107060== possibly lost: 0 bytes in 0 blocks
==107060== still reachable: 0 bytes in 0 blocks
==107060== suppressed: 0 bytes in 0 blocks
[CAUSE]
In btrfs_free_block_groups(), we use
rbtree_postorder_for_each_entry_safe() to iterate all block group cache.
However since we're already doing post order iteration, we shouldn't
call rb_erase() during that iteration, as it would re-balance the tree,
and break the post order iteration.
This wrong rb_erase() call leads to above memory leak.
[FIX]
Kill that wrong rb_erase() call.
Fixes: b1bd3cd93f ("btrfs-progs: reform block groups caches structure")
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
Valgrind reports the following error for fsck/012:
adding new tree backref on start 4206592 len 4096 parent 0 root 5
==100735== Syscall param pwrite64(buf) points to uninitialised byte(s)
==100735== at 0x49F303A: pwrite (in /usr/lib/libpthread-2.31.so)
==100735== by 0x1A5C85: write_extent_to_disk (extent_io.c:815)
==100735== by 0x1B2507: write_and_map_eb (disk-io.c:512)
==100735== by 0x1B26A7: write_tree_block (disk-io.c:545)
==100735== by 0x1D4822: __commit_transaction (transaction.c:148)
==100735== by 0x1D4AA2: btrfs_commit_transaction (transaction.c:213)
==100735== by 0x16360D: fixup_extent_refs (main.c:7662)
==100735== by 0x16449F: check_extent_refs (main.c:8033)
==100735== by 0x166199: check_chunks_and_extents (main.c:8786)
==100735== by 0x166441: do_check_chunks_and_extents (main.c:8842)
==100735== by 0x169D13: cmd_check (main.c:10324)
==100735== by 0x11CDC6: cmd_execute (commands.h:125)
==100735== Address 0x4e8aeb0 is 128 bytes inside a block of size 4,224 alloc'd
==100735== at 0x483BB65: calloc (vg_replace_malloc.c:762)
==100735== by 0x1A54C5: __alloc_extent_buffer (extent_io.c:609)
==100735== by 0x1A5AD1: alloc_extent_buffer (extent_io.c:752)
==100735== by 0x1B1A0A: btrfs_find_create_tree_block (disk-io.c:222)
==100735== by 0x1BD4A2: btrfs_alloc_free_block (extent-tree.c:2538)
==100735== by 0x1A8CE3: __btrfs_cow_block (ctree.c:322)
==100735== by 0x1A91C6: btrfs_cow_block (ctree.c:415)
==100735== by 0x1AB16C: btrfs_search_slot (ctree.c:1185)
==100735== by 0x160BBC: delete_extent_records (main.c:6652)
==100735== by 0x16343F: fixup_extent_refs (main.c:7629)
==100735== by 0x16449F: check_extent_refs (main.c:8033)
==100735== by 0x166199: check_chunks_and_extents (main.c:8786)
==100735==
[CAUSE]
For new extent buffer allocated, we don't initialize its content.
This is not a major concern, at all.
For the above report, the reported range is inside the unused part of
the extent buffer, thus won't cause anything.
Regular btrfs_cow_block() will cover all the used ranges of one extent
buffer.
[FIX]
But still, since kernel initialize the extent buffer with 0, it won't
hurt to do extra initialized to make valgrind happy.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
Valgrind reports the following error for fsck/007, which is only
repairable for original mode:
==97599== Conditional jump or move depends on uninitialised value(s)
==97599== at 0x1D4A42: btrfs_commit_transaction (transaction.c:207)
==97599== by 0x16475C: check_extent_refs (main.c:8097)
==97599== by 0x166199: check_chunks_and_extents (main.c:8786)
==97599== by 0x166441: do_check_chunks_and_extents (main.c:8842)
==97599== by 0x169D13: cmd_check (main.c:10324)
==97599== by 0x11CDC6: cmd_execute (commands.h:125)
==97599== by 0x11D712: main (btrfs.c:386)
==97599==
[CAUSE]
If btrfs_write_dirty_block_groups() get called with no block group
dirtied (no dirty extents created), the return value of it is
uninitialized, as the stack @ret is not initialized at all.
[FIX]
Initialize @ret to 0 for btrfs_write_dirty_block_groups() as if there is
no dirty block groups, we do nothing and shouldn't fail.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
Valgrind reports the following error for fsck/002 (which only supports
original mode):
==97088== Conditional jump or move depends on uninitialised value(s)
==97088== at 0x15BFF6: add_data_backref (main.c:4884)
==97088== by 0x16025C: run_next_block (main.c:6452)
==97088== by 0x165539: deal_root_from_list (main.c:8471)
==97088== by 0x166040: check_chunks_and_extents (main.c:8753)
==97088== by 0x166441: do_check_chunks_and_extents (main.c:8842)
==97088== by 0x169D13: cmd_check (main.c:10324)
==97088== by 0x11CDC6: cmd_execute (commands.h:125)
==97088== by 0x11D712: main (btrfs.c:386)
[CAUSE]
In alloc_data_backref(), only ref->node is set to 0.
While ref->disk_bytenr is not initialized at all.
And then in add_data_backref(), if @back is a newly allocated data
backref, we use the garbage from back->disk_bytenr to determine if we
should reset them.
[FIX]
Fix it by initialize the whole data_backref structure in
alloc_data_backref().
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
