Commit graph

193 commits

Author SHA1 Message Date
Qu Wenruo 2b1be545db btrfs-progs: remove the duplicated @level parameter for btrfs_bin_search()
We can easily get the level from @eb parameter, thus the level is not
needed.

This is inspired by the work of Marek in U-boot.

Cc: Marek Behun <marek.behun@nic.cz>
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-05-04 20:49:01 +02:00
David Sterba 8c85b34420 btrfs-progs: move backref.[ch] to kernel-shared/
The files are very close to kernel versions, for that keep them in the
shared directory.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-05-04 20:48:35 +02:00
Qu Wenruo 4aa0dcba8c btrfs-progs: do proper error handling in add_cache_extent()
If we have memory allocation failure in add_cache_extent(), it will
simply exit with one error message.

That's definitely not proper, especially when all but one call sites
have handled the error.

This patch will return -ENOMEM for add_cache_extent(), and fix the only
call site which doesn't handle error from it.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-05-04 20:48:35 +02:00
Qu Wenruo 6384f9df92 btrfs-progs: check: sanitize the return value for qgroup error
[BUG]
btrfs check can return strange return value for shell:
 [Inferior 1 (process 48641) exited with code 0213]
					      ^^^^

[CAUSE]
It's caused by the incorrect handling of qgroup error.

qgroup_report_ret can be -117 (-EUCLEAN), using that value with exit()
can cause overflow, causing return value not properly recognized.

[FIX]
Fix it by sanitize the return value to 0 or 1.

Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-04-01 19:56:42 +02:00
Adam Borowski 3d379b1341 btrfs-progs: lots of typo fixes (codespell)
Signed-off-by: Adam Borowski <kilobyte@angband.pl>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:38 +02:00
Adam Borowski ac405aa597 btrfs-progs: check: typo in an error message: "boudnary"
Reviewed-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: Adam Borowski <kilobyte@angband.pl>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:37 +02:00
Qu Wenruo 3c2dbee299 btrfs-progs: check/original: Fix uninitialized memory for newly allocated data_backref
[BUG]
Valgrind reports the following error for fsck/002 (which only supports
original mode):
  ==97088== Conditional jump or move depends on uninitialised value(s)
  ==97088==    at 0x15BFF6: add_data_backref (main.c:4884)
  ==97088==    by 0x16025C: run_next_block (main.c:6452)
  ==97088==    by 0x165539: deal_root_from_list (main.c:8471)
  ==97088==    by 0x166040: check_chunks_and_extents (main.c:8753)
  ==97088==    by 0x166441: do_check_chunks_and_extents (main.c:8842)
  ==97088==    by 0x169D13: cmd_check (main.c:10324)
  ==97088==    by 0x11CDC6: cmd_execute (commands.h:125)
  ==97088==    by 0x11D712: main (btrfs.c:386)

[CAUSE]
In alloc_data_backref(), only ref->node is set to 0.
While ref->disk_bytenr is not initialized at all.

And then in add_data_backref(), if @back is a newly allocated data
backref, we use the garbage from back->disk_bytenr to determine if we
should reset them.

[FIX]
Fix it by initialize the whole data_backref structure in
alloc_data_backref().

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:37 +02:00
Qu Wenruo 0a472ef887 btrfs-progs: check/original: Fix uninitialized stack memory access for deal_root_from_list()
[BUG]
With valgrind, fsck/002 test with original mode would report the
following valgrind error:
  ==90600== Conditional jump or move depends on uninitialised value(s)
  ==90600==    at 0x15C280: pick_next_pending (main.c:4949)
  ==90600==    by 0x15F3CF: run_next_block (main.c:6175)
  ==90600==    by 0x1655CC: deal_root_from_list (main.c:8486)
  ==90600==    by 0x1660C7: check_chunks_and_extents (main.c:8762)
  ==90600==    by 0x166439: do_check_chunks_and_extents (main.c:8842)
  ==90600==    by 0x169D0B: cmd_check (main.c:10324)
  ==90600==    by 0x11CDC6: cmd_execute (commands.h:125)
  ==90600==    by 0x11D712: main (btrfs.c:386)

[CAUSE]
The problem happens like this:
deal_root_from_list(@list is empty)
|- stack @last is not initialized
|- while(!list_empty(list)) {} is skipped
|- run_next_block(&last);
   |- pick_next_pending(*last);
      |- node_start = last;

Since the stack @last is not initialized in deal_root_from_list(), the
final node_start = last assignment would just fetch the garbage from
stack.

[FIX]
Fix the problem by initializing @last to 0, as that's exactly what the
first while loop did.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:37 +02:00
Qu Wenruo 3a2d040070 btrfs-progs: check/lowmem: Fix access on uninitialized memory
[BUG]
There are some reports on fsck/001 test segfault failure with lowmem mode.

While I failed to reproduce it, valgrind still catches it with the
following output:

  Delete backref in extent [12845056 1048576]
  ERROR: file extent [257, 0] has unaligned disk bytenr: 755944791, should be aligned to 4096
  ERROR: file extent[257 0] root 5 owner 5 backref lost
  Deleted root 5 item[257, 108, 0]
  ==29080== Conditional jump or move depends on uninitialised value(s)
  ==29080==    at 0x1A81D7: btrfs_release_path (ctree.c:97)
  ==29080==    by 0x192C33: repair_extent_data_item (mode-lowmem.c:3330)
  ==29080==    by 0x1962FF: check_leaf_items (mode-lowmem.c:4696)
  ==29080==    by 0x196ABF: walk_down_tree (mode-lowmem.c:4858)
  ==29080==    by 0x197762: check_btrfs_root (mode-lowmem.c:5157)
  ==29080==    by 0x198335: check_chunks_and_extents_lowmem (mode-lowmem.c:5450)
  ==29080==    by 0x166414: do_check_chunks_and_extents (main.c:8829)
  ==29080==    by 0x169CF7: cmd_check (main.c:10313)
  ==29080==    by 0x11CDC6: cmd_execute (commands.h:125)
  ==29080==    by 0x11D712: main (btrfs.c:386)
  ==29080==

[CAUSE]
In repair_extent_data_item() if we find unaligned file extent, we just
delete it and kick in hole punch procedure.

The problem is, file extent deletion is done before initializing @path.
And when the deletion is done without problem, we will goto out tag,
which will release @path, containing uninitialized values, and
triggering segfault.

[FIX]
Don't try to abort trans nor free path if we're going through file
extent deletion routine.

Fixes: 0617bde3bc ("btrfs-progs: lowmem: delete unaligned bytes extent data under repair")
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:36 +02:00
Marcos Paulo de Souza 59af046fd2 btrfs-progs: qgroup-verify: Remove duplicated message in report_qgroups
Since 1d5b2ad9 ("btrfs-progs: qgroup-verify: Don't treat qgroup
difference as error if the fs hasn't initialized a rescan") a new
message is being printed when the qgroups is incosistent and the rescan
hasn't being executed, so remove the later message send to stderr.

While in this function, simplify the check for a not executed rescan
since !counts.rescan_running and counts.rescan_running == 0 means the
same thing.

Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:36 +02:00
Josef Bacik 1173a4ba26 btrfs-progs: fix hole error output in fsck
If we don't find holes in our hole rb tree we'll just assume there's a
gap from 0 to the length of the file and print that out.  But this
simply isn't correct, we could have a gap between the last extent and
the isize, or 0 and the start of the first extent.  Fix the error
message to tell us exactly where the hole is.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:35 +02:00
Josef Bacik f2832d534b btrfs-progs: fix lowmem check's handling of holes
Lowmem check had the opposite problem of normal check, it caught gaps
that started at 0, but would still fail with my fixes in place.  This is
because lowmem check doesn't take into account the isize of the inode.
Address this by making sure we do not complain about gaps that are after
isize.  This makes lowmem pass with my fixes applied, and still fail
without my fixes.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:35 +02:00
Josef Bacik 9123fc6142 btrfs-progs: fix check to catch gaps at the start of the file
When writing my test for the i_size patches, I noticed that I was not
actually failing without my patches as I should have been.  This is
because we only check if the inode record extent end is < isize, we
don't check if the inode record extent start is > 0.  Add this check to
make sure we're catching holes that start at the beginning of the file.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:35 +02:00
David Sterba 2efe160bc7 btrfs-progs: move name hashing functions to ctree.h and delete hash.h
The two functions are trivial and kernel has them in ctree.h, so do the
same and remove hash.h.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:35 +02:00
David Sterba 58bcd4260f btrfs-progs: move free-space-tree.[ch] to kernel-shared/
The files are very close to kernel versions, for that keep them in the
shared directory.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:34 +02:00
Su Yue 162d891e4a btrfs-progs: pass @trans to functions working with dirty block groups
We are going to touch dirty_bgs in transaction directly, so every call
chain should pass @trans to the leaf functions.

Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Su Yue <Damenly_Su@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-03 19:58:54 +01:00
Qu Wenruo b3621da146 btrfs-progs: check/original: Detect invalid extent generation
Much like what we have done in lowmem mode, also detect and report
invalid extent generation in original mode.

Unlike lowmem mode, we have extent_record::generation, which is the
higher number of generations in EXTENT_ITEM, EXTENT_DATA or tree block
header, so there is no need to check generations in different locations.

For repair, we still need to depend on --init-extent-tree, as directly
modifying extent items can easily cause conflicts with delayed refs,
thus it should be avoided.

Reviewed-by: Su Yue <Damenly_Su@gmx.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-01-09 14:27:10 +01:00
Qu Wenruo 918ba847bc btrfs-progs: check/lowmem: Detect invalid EXTENT_ITEM and EXTENT_DATA generation
Since older `btrfs check --init-extent-tree` could cause invalid
EXTENT_ITEM generation for data extents, add such check to lowmem mode
check.

Also add such generation check to file extents too.

For the repair part, I don't have any good idea yet. So affected user
may depend on --init-extent-tree again.

Reviewed-by: Su Yue <Damenly_Su@gmx.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-01-09 14:27:10 +01:00
Qu Wenruo 8d45dc270a btrfs-progs: check: Populate extent generation correctly for data extents
[BUG]
When using `btrfs check --init-extent-tree`, we will create incorrect
generation number for data extents in extent tree:

        item 10 key (13631488 EXTENT_ITEM 1048576) itemoff 15828 itemsize 53
                refs 1 gen 0 flags DATA
                extent data backref root FS_TREE objectid 257 offset 0 count 1

[CAUSE]
Since data extent generation is not as obvious as tree blocks, which has
header containing its generations, so for data extents, its
extent_record::generation is not really updated, resulting such 0
generation.

[FIX]
To get generation of a data extent, there are two sources we can rely:
- EXTENT_ITEM
  There is always a btrfs_extent_item::generation can be utilized.
  Although this is not possible for --init-extent-tree use case.

- EXTENT_DATA
  We have btrfs_file_extent_item::generation for regular and
  preallocated data extents.
  Since --init-extent-tree will go through subvolume trees, this would
  be the main source for extent data generation.

Then we only need to make add_data_backref() to accept @gen parameter,
and pass it down to extent_record structure.

And for the final extent item generation update, here we add extra
fallback values, if we can't find FILE_EXTENT items.
In that case, we just fall back to current transid.

With this modification, recreated data EXTENT_ITEM now has correct
generation number:

        item 10 key (13631488 EXTENT_ITEM 1048576) itemoff 15828 itemsize 53
                refs 1 gen 6 flags DATA
                extent data backref root FS_TREE objectid 257 offset 0 count 1

Reviewed-by: Su Yue <Damenly_Su@gmx.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-01-09 14:27:10 +01:00
Qu Wenruo 9a85b7db02 btrfs-progs: check: Initialize extent_record::generation member
[BUG]
When using `btrfs check --init-extent-tree`, there is a pretty high
chance that the result fs can't pass tree-checker:

  BTRFS critical (device dm-3): corrupt leaf: block=5390336 slot=149 extent bytenr=20115456 len=4096 invalid generation, have 16384 expect (0, 360]
  BTRFS error (device dm-3): block=5390336 read time tree block corruption detected
  BTRFS error (device dm-3): failed to read block groups: -5
  BTRFS error (device dm-3): open_ctree failed

[CAUSE]
The result fs has a pretty screwed up EXTENT_ITEMs for data extents:

        item 148 key (20111360 EXTENT_ITEM 4096) itemoff 8777 itemsize 53
                refs 1 gen 0 flags DATA
                extent data backref root FS_TREE objectid 841 offset 0 count 1
        item 149 key (20115456 EXTENT_ITEM 4096) itemoff 8724 itemsize 53
                refs 1 gen 16384 flags DATA
                extent data backref root FS_TREE objectid 906 offset 0 count 1

Kernel tree-checker will accept 0 generation, but that 16384 generation
is definitely going to trigger the alarm.

Looking into the code, it's add_extent_rec_nolookup() allocating a new
extent_rec, but not copying all members from parameter @tmpl, resulting
generation not properly initialized.

[FIX]
Just copy tmpl->generation in add_extent_rec_nolookup(). And since all
call sites have set all members of @tmpl to 0 before
add_extent_rec_nolookup(), we shouldn't get garbage values.

For the 0 generation problem, it will be solved in another patch.

Issue: #225 (Not the initial report, but extent tree rebuild result)
Reviewed-by: Su Yue <Damenly_Su@gmx.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-01-09 14:27:10 +01:00
Qu Wenruo 3dcce48fd7 btrfs-progs: check/original: Do extra verification on file extent item
[BUG]
For certain fuzzed image, `btrfs check` will fail with the following
call trace:

  Checking filesystem on issue_213.raw
  UUID: 99e50868-0bda-4d89-b0e4-7e8560312ef9
  [1/7] checking root items
  [2/7] checking extents
  Program received signal SIGABRT, Aborted.
  0x00007ffff7c88f25 in raise () from /usr/lib/libc.so.6
  (gdb) bt
  #0  0x00007ffff7c88f25 in raise () from /usr/lib/libc.so.6
  #1  0x00007ffff7c72897 in abort () from /usr/lib/libc.so.6
  #2  0x00005555555abc3e in run_next_block (...) at check/main.c:6398
  #3  0x00005555555b0f36 in deal_root_from_list (...) at check/main.c:8408
  #4  0x00005555555b1a3d in check_chunks_and_extents (fs_info=0x5555556a1e30) at check/main.c:8690
  #5  0x00005555555b1e3e in do_check_chunks_and_extents (fs_info=0x5555556a1e30) a
  #6  0x00005555555b5710 in cmd_check (cmd=0x555555696920 <cmd_struct_check>, argc
  #7  0x0000555555568dc7 in cmd_execute (cmd=0x555555696920 <cmd_struct_check>, ar
  #8  0x0000555555569713 in main (argc=2, argv=0x7fffffffde70) at btrfs.c:386

[CAUSE]
This fuzzed images has a corrupted EXTENT_DATA item in data reloc tree:
        item 1 key (256 EXTENT_DATA 256) itemoff 16111 itemsize 12
                generation 0 type 2 (prealloc)
                prealloc data disk byte 16777216 nr 0
                prealloc data offset 0 nr 0

There are several problems with the item:
- Bad item size
  12 is too small.
- Bad key offset
  offset of EXTENT_DATA type key represents file offset, which should
  always be aligned to sector size (4K in this particular case).

[FIX]
Do extra item size and key offset check for original mode, and remove
the abort() call in run_next_block().

And to show off how robust lowmem mode is, lowmem can handle it without
any hiccup.

With this fix, original mode can detect the problem properly:
  Checking filesystem on issue_213.raw
  UUID: 99e50868-0bda-4d89-b0e4-7e8560312ef9
  [1/7] checking root items
  [2/7] checking extents
  ERROR: invalid file extent item size, have 12 expect (21, 16283]
  ERROR: errors found in extent allocation tree or chunk allocation
  [3/7] checking free space cache
  [4/7] checking fs roots
  root 18446744073709551607 root dir 256 error
  root 18446744073709551607 inode 256 errors 62, no orphan item, odd file extent, bad file extent
  ERROR: errors found in fs roots
  found 131072 bytes used, error(s) found
  total csum bytes: 0
  total tree bytes: 131072
  total fs tree bytes: 32768
  total extent tree bytes: 16384
  btree space waste bytes: 124774
  file data blocks allocated: 0
   referenced 0

Issue: #213
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Su Yue <Damenly_Su@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-01-09 14:27:09 +01:00
Qu Wenruo 36128dff44 btrfs-progs: check/lowmem: Fix a false alert on uninitialized value
[BUG]
When compiling the devel branch with commit fb8f05e40b458
("btrfs-progs: check: Make repair_imode_common() handle inodes in
subvolume trees"), the following warning will be reported:

  check/mode-common.c: In function ‘detect_imode’:
  check/mode-common.c|1071 col 23| warning: ‘imode’ may be used uninitialized in this function [-Wmaybe-uninitialized]
  1071 |   *imode_ret = (imode | 0700);
       |                ~~~~~~~^~~~~~~

This only occurs for regular build. If compiled with D=1, the warning
just disappears.

[CAUSE]
Looks like a bug in gcc optimization.
The code will only set @imode_ret when @found is true.
And for every "found = true" assignment we have assigned @imode.
So this is just a false alert.

[FIX]
I hope I can fix the problem of GCC, but obviously I can't (at least for
now).

So let's assign an initial value 0 to @imode to suppress the false
alert.

Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-22 19:09:51 +01:00
Johannes Thumshirn e388bf386b btrfs-progs: check: warn users about the possible dangers of --repair
The manual page of btrfsck clearly states 'btrfs check --repair' is a
dangerous operation.

Although this warning is in place users do not read the manual page
and/or are used to the behaviour of fsck utilities which repair the
filesystem, and thus potentially cause harm.

Similar to 'btrfs balance' without any filters, add a warning and a
countdown, so users can bail out before eventual corrupting the
filesystem more than it already is.

To override the timeout, let --force skip it and continue.

Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-22 19:07:20 +01:00
Qu Wenruo 989a99b5f8 btrfs-progs: Replace btrfs_block_group_cache::item with dedicated members
We access btrfs_block_group_cache::item mostly for @used and @flags.

@flags is already a dedicated member in btrfs_block_group_cache, only
@used doesn't have a dedicated member.

This patch will remove btrfs_block_group_cache::item and add
btrfs_block_group_cache::used.

It's the btrfs-progs equivalent of the following kernel patches:
btrfs: move block_group_item::used to block group
btrfs: move block_group_item::flags to block group
btrfs: remove embedded block_group_cache::item

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:21:09 +01:00
Qu Wenruo e46281d6fb btrfs-progs: Refactor excluded extent functions to use fs_info
The following functions are just using @root to reach fs_info:
- exclude_super_stripes
- free_excluded_extents
- add_excluded_extent

Refactor them to use fs_info directly.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:21:08 +01:00
Qu Wenruo b9ea7c1b23 btrfs-progs: check/original: Add check and repair for invalid inode generation
There are at least two bug reports of kernel tree-checker complaining
about invalid inode generation.

All offending inodes seem to be caused by old kernel around 2014, with
inode generation overflow.

So add such check and repair ability to lowmem mode check first.

This involves:

- Calculate the inode generation upper limit
  Unlike the lowmem mode context, we don't have anyway to determine if
  this inode belongs to log tree.
  So we use super_generation + 1 as upper limit, just like what we did
  in kernel tree checker.

- Check if the inode generation is larger than the upper limit

- Repair by resetting inode generation to current transaction
  generation
  The difference is, in original mode, we have a common trans handle for
  all repair and reset path for each repair.

Reported-by: Charles Wright <charles.v.wright@gmail.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Tested-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:21:07 +01:00
Qu Wenruo 0a6e40bdc1 btrfs-progs: check/lowmem: Add check and repair for invalid inode generation
There are at least two bug reports of kernel tree-checker complaining
about invalid inode generation.

All offending inodes seem to be caused by old kernel around 2014, with
inode generation overflow.

So add such check and repair ability to lowmem mode check first.

This involves:

- Calculate the inode generation upper limit
  If it's an inode from log tree, then the upper limit is
  super_generation + 1, otherwise it's super_generation.

- Check if the inode generation is larger than the upper limit

- Repair by resetting inode generation to current transaction
  generation

Reported-by: Charles Wright <charles.v.wright@gmail.com>
Tested-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:21:07 +01:00
Qu Wenruo eae0c8e32f btrfs-progs: check/original: Fix inode mode in subvolume trees
To make original mode to repair imode error in subvolume trees, this
patch will do:

- Remove the show-stopper checks for root->objectid.
  Now repair_imode_original() will accept inodes in subvolume trees.

- Export detect_imode() for original mode
  Due to the call requirement, original mode must use an existing trans
  handler to do the repair, thus we need to re-implement most of the
  work done in repair_imode_common().

- Make repair_imode_original() to use detect_imode().

- Free the path after reset_imode()
  reset_imode() keeps the path, as lowmem mode uses path to locate its
  current check position.
  But for original mode, the unreleased path can cause later repair to
  report warning, so we need to manually release the path.

- Update rec->imode after imode reset
  So later repair depending on rec->imode can get correct value.

- Move the repair before repair_inode_nlinks()
  repair_inode_nlinks() needs correct imode to add DIR_INDEX/DIR_ITEM.
  So moving the repair before repair_inode_nlinks() makes the latter
  repair happier.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:21:07 +01:00
Qu Wenruo 69eac9de0c btrfs-progs: check/lowmem: Repair bad imode early
For lowmem mode, if we hit a bad inode mode, normally it is reported
when we checking the DIR_INDEX/DIR_ITEM of the parent inode.

If we didn't repair at that time, the error will be recorded even if we
fixed it later.

So this patch will check for INODE_ITEM_MISMATCH error type, and if it's
really caused by invalid imode, repair it and clear the error.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:21:07 +01:00
Qu Wenruo 7f8383b7a6 btrfs-progs: check: Make repair_imode_common() handle inodes in subvolume trees
[[PROBLEM]]
Before this patch, repair_imode_common() can only handle two types of
inodes:

- Free space cache inodes
- ROOT DIR inodes

For inodes in subvolume trees, the core complexity is how to determine
the correct imode, thus it was not implemented.

However there are more reports of incorrect imode in subvolume trees, we
need to support such fix.

[[ENHANCEMENT]]
So this patch adds a new function, detect_imode(), to detect imode for
inodes in subvolume trees.  The policy here is, try our best to find a
valid imode to recovery.  If no convicing info can be found, fail out.

That function will determine imode by:

1) Search for INODE_REF of the inode
   If we have INODE_REF, we will then try to find DIR_ITEM/DIR_INDEX.
   As long as one valid DIR_ITEM or DIR_INDEX can be found, we convert
   the BTRFS_FT_* to imode, then call it a day.
   This should be the most accurate way.

2) Search for DIR_INDEX/DIR_ITEM belongs to this inode
   If above search fails, we falls back to locate the DIR_INDEX/DIR_ITEM
   just after the INODE_ITEM.
   Thus this only works for non-empty directory.
   If any can be found, it's definitely a directory.

3) Search for EXTENT_DATA belongs to this inode
   If EXTENT_DATA can be found, it's either REG or LNK.
   Thus this only works for non-empty file or soft link.
   For this case, we default to REG, as user can inspect the file to
   determine if it's a file or just a path.

4) Use rdev to detect BLK/CHR
   If all above fails, but INODE_ITEM has non-zero rdev, then it's either
   a BLK or CHR file. Then we default to BLK.

5) Fail out if none of above methods succeeded
   No educated guess to make things worse.

[[SHORTCOMING]]
The above search is not perfect, there are cases where we can't repair:
E.g. orphan empty regular inode.  Since it's already orphan, it has no
INODE_REF. And it's regular empty file, it has no DIR_INDEX nor
EXTENT_DATA nor rdev. Thus we can't recover.  Although for this case, it
really doesn't matter as it's already orphan and will be deleted anyway.

Furthermore, due to the DIR_ITEM/DIR_INDEX/INODE_REF repair code which
can happen before imode repair, it's possible that DIR_ITEM search code
may not be executed.  If there is only DIR_ITEM remaining, repair code
will remove the DIR_ITEM completely and move the inode to lost+found,
leaving us no info to rebuild imode.  If there is DIR_INDEX missing,
repair code will re-insert the DIR_INDEX, then imode repair code will go
DIR_INDEX directly.

But overall, the repair code should handle the invalid imode caused by
older kernels without problem.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:21:07 +01:00
Qu Wenruo ac9e07a780 btrfs-progs: check: find imode using info from INODE_REF item
Introduce a function, find_file_type(), to find filetype using info from
INODE_REF, including dir_id from key index/name from inode_ref_item.

This function will:

- Search DIR_INDEX first
  DIR_INDEX is easier since there is only one item in it.

- Validate the DIR_INDEX item
  If the DIR_INDEX is valid, use the filetype and call it a day.

- Search DIR_ITEM then
  It needs extra iteration since it's possible to have hash collision.

- Validate the DIR_ITEM
  If valid, call it a day. Or return -ENOENT;

This would be used as the primary method to determine the imode in later
imode repair code.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:21:07 +01:00
Qu Wenruo 87207654f1 btrfs-progs: check: Export btrfs_type_to_imode
This function will be later used by common mode code, so export it.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:21:07 +01:00
Rosen Penev 5d72055066 btrfs-progs: Fix printf formats
Discovered with cppcheck. Fix signed/unsigned int mismatches, sizeof and
long formats.

Pull-request: #197
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-10-14 17:31:05 +02:00
Johannes Thumshirn ed33908b44 btrfs-progs: update checksumming api
Update the checksumming API to be able to cope with more checksum types
than just CRC32C. The finalization call is merged into btrfs_csum_data.

There are some fixme's and asserts added that need to be resolved.

Co-developed-by: David Sterba <dsterba@suse.com>
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-10-14 17:28:34 +02:00
Johannes Thumshirn 7b4f1035a6 btrfs-progs: pass checksum type to btrfs_csum_data()/btrfs_csum_final()
In preparation to supporting new checksum algorithm pass the checksum type
to btrfs_csum_data/btrfs_csum_final, this allows us to encapsulate any
differences in processing into the respective functions

Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-10-14 17:28:28 +02:00
Johannes Thumshirn de68086e35 btrfs-progs: don't assume checksums are always 4 bytes
Pass pointer to a generic buffer instead of fixed size that crc32c
currently uses.

Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-10-14 17:28:25 +02:00
David Sterba 0889161127 btrfs-progs: check: move device_record to main.c
The structure is used only inside main.c, no need to export it.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-10-14 17:27:41 +02:00
David Sterba 705f4ecf02 btrfs-progs: check: remove flat include switch from common.h
The header will be used only internally, so the FLAT_INCLUDES is not
needed.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-10-14 17:27:39 +02:00
David Sterba 1b42e71053 btrfs-progs: copy btrfsck.h to check/common.h
We want just one header for the check API (similar to what mkfs does)
but as btrfsck.h is exported header (libbtrfs), it needs some
deprecation beriod before it's moved through there are probably no users
of that header file in particular.

Copy the header to check, all modifications and cleanups won't affect
the public header.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-10-14 17:27:37 +02:00
David Sterba 8e68287d88 btrfs-progs: move qgroup-verify.[ch] to check/
The only user of the functionality is in check.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-10-14 17:27:35 +02:00
Qu Wenruo c31bd8db54 btrfs-progs: check/lowmem: Skip nbytes check for orphan inodes
For orphan inodes, kernel won't update its nbytes and size since it's a
waste of time.

So lowmem check can report false alert on some orphan inodes.

Fix it by checking if the inode is an orphan before
complaining/repairing its nbytes.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-09-04 16:08:56 +02:00
Josef Bacik 73ff25c65d btrfs-progs: check: don't check nbytes on unlinked files
We don't update the inode when evicting it, so the nbytes will be wrong
in between transaction commits.  This isn't a problem, stop complaining
about it to make generic/269 stop randomly failing. The orphan outdated
inodes can be still present but check will not skip them.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-09-04 16:08:11 +02:00
Qu Wenruo 1baf729743 btrfs-progs: check/original: Check and repair root item geneartion
Add such ability to original mode to fix root generation mismatch, which
can be rejected by kernel.

Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-09-04 16:06:52 +02:00
Qu Wenruo ebf29e39c3 btrfs-progs: check/lowmem: Check and repair root generation
Since kernel is going to reject any root item which is newer than super
block generation, we need to provide a way to fix such problem in
btrfs-check.

This patch addes the ability to report and repair root generation in
lowmem mode.

This is done by cowing the root node, so we will update the root
generation along with the root node generation at commit transaction
time.

Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-09-04 16:06:50 +02:00
Adam Borowski 90a87e817d btrfs-progs: check: fix option parsing for -E
The option -E has a mandatory argument that was missing from the short
option spec, thus it always crashed.

Signed-off-by: Adam Borowski <kilobyte@angband.pl>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-09-04 16:06:24 +02:00
Naohiro Aota 8ca6c0c3c7 btrfs-progs: check: initialize qgroup_item_count in earlier stage
"btrfsck -Q" segfaults because it does not call qgroup_set_item_count_ptr()
properly:

  # btrfsck -Q /dev/sdk
  Opening filesystem to check...
  Checking filesystem on /dev/sdk
  UUID: 34a35bbc-43f8-40f0-8043-65ed33f2e6c3
  Print quota groups for /dev/sdk
  UUID: 34a35bbc-43f8-40f0-8043-65ed33f2e6c3
  Segmentation fault (core dumped)

Since "struct task_ctx ctx" is global, we can just move
qgroup_set_item_count_ptr() much earlier stage in the check process to
avoid to forget initializing it.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-26 17:46:42 +02:00
Josef Bacik 42a1aaeec4 btrfs-progs: deal with drop_progress properly in fsck
While testing snapshot deletion with dm-log-writes I saw that I was
failing the fsck sometimes when the fs was actually in the correct
state.  This is because we only skip blocks on the same level of
root_item->drop_level.  If the drop_level < the root level then we could
very well walk into nodes that we wouldn't actually walk into on fs
mount, because the drop_progress is further ahead in the slot of the
root.  Instead only process the slots of the nodes that are above the
drop_progress key.  With this patch in place we no longer improperly
fail to check fs'es that have a drop_progress set with a drop_level <
root level.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-26 17:46:42 +02:00
David Sterba c8bea2b84b btrfs-progs: move rbtree-utils.[ch] to common/
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:04 +02:00
David Sterba c07960c8be btrfs-progs: move utils.[ch] to common/
Update include paths and remove some duplicates.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:04 +02:00
David Sterba aac564aca6 btrfs-progs: move commonh to common/
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:03 +02:00