The filesystem existence on a device is manifested by the signature,
during the mkfs process we write it first and then create other
structures. Such filesystem is not valid and should not be registered
during device scan nor listed among devices from blkid.
This patch will introduce two staged creation. In the first phase, the
signature is wrong, but recognized as a partially created filesystem (by
open or scan helpers). Once we successfully create and write everything,
we fixup the signature. At this point automated scanning should find
a valid filesystem on all devices.
We can also rely on the partially created filesystem to do better error
handling during creation. We can just bail out and do not need to clean
up.
The partial signature is '!BHRfS_M', can be shown by
btrfs inspect-internal dump-super -F image
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We call scan ioctl on the devices too early, when most of the filesystem
structures are not yet created. Move the registration to the end, after
the filesystem gets closed.
Signed-off-by: David Sterba <dsterba@suse.com>
Currently the superblock is created first, with a valid signaure, but
the rest of the filesystem is missing. When the creation process is
interrupted, the filesystem still might be considered as valid.
To prevent that, create the filesytem with an invalid signature that
would be still recognized during the mkfs process, and finalize at the
end.
Signed-off-by: David Sterba <dsterba@suse.com>
As we're passing a set of flags, the enum type is not appropriate.
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add options to show file and line or stack trace for error/warning
messages that use the common helpers. Possible to let any error stop
execution for ease of analysis and debugging.
Signed-off-by: David Sterba <dsterba@suse.com>
Old convert codes uses both 0400 permission and INODE_READONLY flag to
make the converted ext2 image readonly.
While new convert treat the inode just as normal inode, with no special
inode flag and uses 0600 permission.
This makes user able to modify converted image unintentionally and make
rollback fails.
This test case will test the regression.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The new convert treats the convert image as a normal file, without any
special flags and permissions.
This is different from original code:
1) Permission changed from 0400 to 0600
2) Inode lacks READONLY flag
This makes we can read-write mount the ext2 image and cause rollback
failure.
Follow old code behavior, use 0400 permission and add back READONLY
flag to fix it.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Do not use fprintf, adjust messages, add verbose errno or at least the
errorr code if there's no clear mapping to a string.
Signed-off-by: David Sterba <dsterba@suse.com>
Change the single-purpose option --low-memory to a generic option that
takes the mode. Currently supported are the original mode and the
low-memory in the same way.
Signed-off-by: David Sterba <dsterba@suse.com>
Introduce a new fsck mode: low memory mode.
Old btrfsck is working efficiently but uses some memory for each extent
item. This method will ensure extents are only iterated once at
extent/chunk tree check process.
But since it uses some memory for each extent item, for a large fs with
several TB metadata, this can easily eat up memory and cause OOM.
To handle such limitation and improve scalability, the new low-memory
mode will not use any heap memory to record which extent is checked.
Instead it will use extent backref to avoid most of uneeded checks on
shared fs/subvolume tree blocks.
And with the use forward and backward reference cross check, we can also
ensure every tree block is at least checked once.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Introduce a new function traverse_tree_block() to do pre-order
traversal, to co-operate with new fs/subvolume tree skip function.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Introduce function should_check() to reduced duplicated tree block check
for fs/subvolume tree.
The idea is, we only check the fs/subvolue tree block if we have the
lowest referencer rootid, according to extent tree.
In that case, we can skip a lot of fs/subvolume tree block checks if
there are a lot of snapshots.
Although we will do a lot of extent tree searches for it.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Introduce an entry function, check_leaf_items() to check all
known/valuable items and update related accounting like total_bytes and
csum_bytes.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Introduce function check_chunk_item() to check a chunk item.
It will check all chunk stripes with dev extents and the corresponding
block group item.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Introduce function check_block_group_item() to check a block group item.
It will check the referencer chunk and the used space accounting with
extent tree.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Introduce function check_dev_item() to check used space with dev extent
items.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Introduce function check_dev_extent_item() to find its referencer chunk.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Introduce function check_extent_item() using previously introduced
functions.
With previous function to check referencer and backref, this function
can be quite easy.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Introduce the function check_shared_data_backref() to check the
referencer of a given shared data backref.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Introduce new function check_extent_data_backref() to search referencer
for a given data backref.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Introduce function check_shared_block_backref() to check shared block
ref.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Introduce a new function check_tree_block_backref() to check if a
backref points to correct referencer.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Introduce function query_tree_block_level() to resolve tree block level
by the following method:
1) tree block backref level
2) tree block header level
And only when header level == backref level, and transid matches, it will
return the tree block level.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Introduce a new function check_data_extent_item() to check if the
corresponding data backref exists in extent tree.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Introduce function check_tree_block_ref() to check whether a tree block
has correct backref in extent tree.
Unlike old extent tree check method, we only use search_slot() to search
reference, no extra structure will be allocated in heap to record what we
have checked.
This method may cause a little more IO, but should work for super large
fs without triggering OOM.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In walk_down_tree(), we may call btrfs_lookup_extent_info() for same tree
block many times, obviously unnecessary. Here we define a simple struct to
record whether we already have gotten tree block's refs:
struct node_refs {
u64 bytenr[BTRFS_MAX_LEVEL];
u64 refs[BTRFS_MAX_LEVEL];
};
I fill a disk partition with linux kernel source codes and use below
test script to have performance test.
#!/bin/bash
echo 3 > /proc/sys/vm/drop_caches
for ((i = 0; i < 20; i++)); do
time ./btrfsck /dev/sdc5
done 2>&1 | grep real | awk -F "[ms]" '{run_time += $2} END{print run_time / 20}'
Before this patch, it averagely took 0.8447s for every btrfsck execution,
and with this patch, it averagely took 0.7807s.
Signed-off-by: Wang Xiaoguang <wangxg.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In read_one_chunk(), we may add an empty entry for a missing device.
However, this entry wasn't being added to the dev_list, and so it never
got freed.
Signed-off-by: Justin Maggard <jmaggard@netgear.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The raid6 code matches kernel implementation that also does the
unaligned access. So to keep the code close, add helpers for unaligned
native access and use them. The helpers are local as we don't plan to
use them elsewhere.
Reported-by: Anatoly Pugachev <matorola@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This will not cause unaligned access as the checksum is at the beginning
of btrfs_header and thus aligned to a page, but for clarity use the
helper.
Signed-off-by: David Sterba <dsterba@suse.com>
The message about discard is printed unconditionally and does not
conform to the --quite option eg. in mkfs. Consolidate the operation
flags into one argument and add support for verbosity.
Signed-off-by: David Sterba <dsterba@suse.com>
