Recently we had a scrub use-after-free caused by unaligned chunk
length, although the fix was submitted, we may want to do extra checks
for a chunk's alignment.
This patch adds such check for the starting bytenr and length of a
chunk, to make sure they are properly aligned to 64K stripe boundary.
By default, the check only leads to a warning but is not treated as an
error, as we expect kernel to handle such unalignment without any
problem.
But if the new debug environmental variable,
BTRFS_PROGS_DEBUG_STRICT_CHUNK_ALIGNMENT, is specified, then we will
treat it as an error. So that we can detect unexpected chunks from
btrfs-progs, and fix them before reaching the end users.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Although kernel scrub code has been updated to handle the unaligned
chunk length, there is also no harm if we can allocate data chunk with
both start and length aligned.
This patch handles this by rounding up the end bytenr when allocating
data chunks for the conversion.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
On multi-device filesystems, a scrub limit may be applied to any of the
devices. Ensure that any limit found is not disregarded.
Since it's more intuitive, keep the lowest non-zero limit found, even
though at the present we don't actually use the exact value.
Pull-request: #733
Issue: #727
Fixes: 7e4a235df1 ("btrfs-progs: scrub status: print device speed limit in status if set")
Signed-off-by: Jonas Malaco <jonas@protocubo.io>
Signed-off-by: David Sterba <dsterba@suse.com>
On multi-device filesystems, a scrub limit may be applied to any of the
devices. Ensure that any limit found is not disregarded.
Since it's more intuitive, keep the lowest non-zero limit found, even
though at the present we don't actually use the exact value.
Pull-request: #733
Issue: #727
Fixes: 7e4a235df1 ("btrfs-progs: scrub status: print device speed limit in status if set")
Signed-off-by: Jonas Malaco <jonas@protocubo.io>
Signed-off-by: David Sterba <dsterba@suse.com>
On multi-device filesystems, scrub status should report "some limits
set" if at least one device has a scrub limit set.
However, with btrfs-progs 6.6.3, this was being reported regardless of
whether any limit actually being set:
# sudo btrfs scrub limit /more/butter
UUID: 989129d9-c96f-4d52-9d68-cbb6d9b2c499
Id Limit Path
-- ----- ---------
1 - /dev/sdc1
2 - /dev/sdd1
# sudo btrfs scrub status /more/butter/
UUID: 989129d9-c96f-4d52-9d68-cbb6d9b2c499
Scrub started: Mon Jan 15 02:00:30 2024
Status: running
Duration: 6:23:19
Time left: 0:49:08
ETA: Mon Jan 15 09:12:57 2024
Total to scrub: 9.83TiB
Bytes scrubbed: 8.72TiB (88.64%)
Rate: 397.47MiB/s (some device limits set)
Error summary: no errors found
Fix it by only setting `limit` to the special marker value 1 if at least
one actual limit is found.
Pull-request: #733
Issue: #727
Fixes: 7e4a235df1 ("btrfs-progs: scrub status: print device speed limit in status if set")
Signed-off-by: Jonas Malaco <jonas@protocubo.io>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
There is a report about failed btrfs-convert, which shows the following
error:
corrupt leaf: root=5 block=5001928998912 slot=1 ino=89911763, invalid previous key objectid, have 89911762 expect 89911763
ERROR: failed to copy ext2 inode 89911320: -5
ERROR: error during copy_inodes -5
WARNING: error during conversion, the original filesystem is not modified
[CAUSE]
Above error is triggered when checking the following items inside a
subvolume:
- inode ref
- dir item/index
- file extent
- xattr
This is to make sure these items have correct previous key.
However btrfs-convert is not following this requirement, it always
inserts those items first, then creates a btrfs_inode for it.
Thus it can lead to the error.
This can only happen for large fs, as for most cases we have all these
modified tree blocks cached, thus tree-checker won't be triggered.
But when the tree block cache is not hit, and we have to read from disk,
then such behavior can lead to above tree-checker error.
[FIX]
Make sure we insert the inode item first, then the file extents/dir
items/xattrs. And after the file extents/dir items/xattrs inserted, we
update the existing inode (to update its size and bytes).
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Unlike kernel where tree-checker would provide enough info so later we
can use "btrfs inspect dump-tree" to catch the offending tree block, in
progs we may not even have a btrfs to start "btrfs inspect dump-tree".
E.g during btrfs-convert.
To make later debuging easier, let's call btrfs_print_tree() for every
error we hit inside tree-checker.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Older versions of sphinx require the contents.rst file otherwise the
build fails, while new versions don't need it and use index.rst.
Sphinx error:
master file btrfs-progs/Documentation/contents.rst not found
make[1]: *** [Makefile:37: man] Error 2
make: *** [Makefile:502: build-Documentation] Error 2
This build error is seen on version 1.7.6-3.
To make it work on old and new versions create a placeholder empty file
but make it a phony build target so new sphinx does not see it and
report as not in any TOC.
[ci skip]
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
There are several warnings regarding the absence of an argument for the
code-block directive on some build servers using python3-sphinx 0.2.2-17.
For example:
Making all in Documentation
[SPHINX] man
ch-subvolume-intro.rst:141: WARNING: Error in "code-block" directive:
1 argument(s) required, 0 supplied.
.. code-block::
27 21 0:19 /subv1 /mnt rw,relatime - btrfs /dev/sda rw,space_cache
Etc...
Add the none argument.
[ci skip]
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The test case would check if "btrfs subvolume create":
- Report error on an existing path
- Still report error if mulitple paths are given and one of them already
exists
- For above case, still created a subvolume for the good parameter
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
When try to create a subvolume where the target path already exists, the
"btrfs" command doesn't return error code correctly.
# mkfs.btrfs -f $dev
# mount $dev $mnt
# touch $mnt/subv1
# btrfs subvolume create $mnt/subv1
ERROR: target path already exists: $mnt/subv1
# echo $?
0
[CAUSE]
The check on whether target exists is done by path_is_dir(), if it
returns 0 or 1, it means there is something in that path already.
But unfortunately commit 5aa959fb34 ("btrfs-progs: subvolume create:
accept multiple arguments") only changed the out label, which would
directly return @ret, not updating the return value correctly.
[FIX]
Make sure all error out branch has their @ret manually updated.
Fixes: 5aa959fb34 ("btrfs-progs: subvolume create: accept multiple arguments")
Issue: #730
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The "=" operator should be used as configure may be run by a POSIX shell
at /bin/sh (like dash). Bash recognises "=" too so this retains
compatibility with it.
Pull-request: #721
Signed-off-by: Sam James <sam@gentoo.org>
Signed-off-by: David Sterba <dsterba@suse.com>
To be consistent with the rest of the code the sysfs helper should
return the -errno instead of passing -1 from various syscalls. Update
callers that relied on -1 as the invalid file descriptor.
Signed-off-by: David Sterba <dsterba@suse.com>
Scrubs which complete in under one second may carry a duration rounded
down to zero. This subsequently results in a bytes_per_sec value of
zero, which corresponds to the Rate metric output, causing intermittent
tests/btrfs/282 failures.
This change ensures that Rate reflects any sub-second bytes processed.
Time left and ETA metrics are also affected by this change, in that they
increase to account for (sub-second) bytes_per_sec.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Disseldorp <ddiss@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
Print one message per scrubbed device and also print the limit if set:
$ btrfs scrub start /mnt
scrub started on /mnt, fsid 9ee93131-f680-4d6c-8ca4-a194506e3081 (pid=27257)
Starting scrub on devid 1 (limit 100.00MiB/s)
Signed-off-by: David Sterba <dsterba@suse.com>
The statfs(2) syscall is deprecated by LSB in favor of statvfs(2),
however we can't replace all uses because we still need the
statfs::f_type to determine the filesystem by magic numer.
Signed-off-by: David Sterba <dsterba@suse.com>
The subvolume cleaning is done by polling but it's possible that the
filesystem turns to read-only (as reported), either due to an error
intentionally. In that case the waiting would be indefinite without an
obvious reason.
To fix that check if the filesystem is still writable in each iteration.
Issue: #535
Link: https://github.com/btrfs/fstests/issues/40
Signed-off-by: David Sterba <dsterba@suse.com>
When there's a speed limit set for a device via
/sysfs/fs/btrfs/FSID/devinfo/scrub_speed_max, show it in the scrub status
output like below:
$ btrfs scrub status -d /mnt
...
Rate: 47.98MiB/s (limit 60MiB/s)
...
If the limit is 0 this means unlimited and is not printed.
For a single device filesystem the limit is printed even without '-d' as
it's clear which device limit applies. For multi-device filesysetms,
without any limits nothing is printed, if there at least one device
limit set then the following is printed:
Rate: 36.37MiB/s (some device limits set)
More details with the -d option.
Issue: #531
Signed-off-by: David Sterba <dsterba@suse.com>
If zstd is not compiled in then a stream fails with a generic error
message:
ERROR: unknown compression: 2
Where BTRFS_ENCODED_IO_COMPRESSION_ZSTD is 2 and there's a case for that
but behind the '#if COMPRESSION_ZSTD'.
Signed-off-by: David Sterba <dsterba@suse.com>
With the recent updates to documentation build the theme must be now
installed as a package. Disable building documentation in all workflows
that do functional tests.
Signed-off-by: David Sterba <dsterba@suse.com>
Since we're already directing the end user to use "btrfs rescue
clear-ino-cache" command, there is not much need to support it in
btrfs-check.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Lowmem mode has improved quite a lot since its introduction, for
read-only check it's definitely fine.
For repair mode, both lowmem and original mode are considered dangerous
especially for complex corruptions with unknown cause.
For now lowmem mode is only bad at fixing fundamentally corrupted cases,
like bad shift offsets or transid, which in real world it's not an easy
repair for the original mode either.
This patch would move the --mode option out of the dangerous section and
update the notes for the lowmem mode on its limitation.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Currently the path of deleted subvolume is printed, we should also print
the numeric id as it's another identifier commonly found and can be used
for a cross reference. In connection with the qgroup deletion it's
making the output clear:
...
Delete subvolume 258 (no-commit): '/mnt/subv1'
Delete qgroup 0/258
...
Signed-off-by: David Sterba <dsterba@suse.com>
The 0/subvolid qgroups are not automatically deleted when the subvolume
is deleted, for historical reasons. There's a command to clean up all
such stale qgroups (btrfs qgroup clean-stale) but this should be also
possible with the subvolume deletion.
With the options we can switch the default to delete the qgroup by
default eventually, if somebody depends on the not deleting behaviour
the negation option can be used.
Issue: #366
Signed-off-by: David Sterba <dsterba@suse.com>
Compiling with clang on aarch64 leads to an error when detecting the
SIMD instruction support. Gcc ignores the arch/feature mismatch.
Conditionally detect the -m flags only on x86_64.
Issue: #712
Signed-off-by: David Sterba <dsterba@suse.com>
The enqueue option should let the user know that the expected operation
hasn't started yet and that it's waiting for another one. Although the
exclusive operations can take long, the two reason should be
distinguished.
Signed-off-by: David Sterba <dsterba@suse.com>
Reported on IRC, that it's unexpected that passing several devices on
command line for 'btrfs device delete' still uses some of the devices
during deletion. The expectation was that they'd be removed at once (and
thus not used for the intermediate chunk relocation).
As it works now, the ioctl removes only one device. As a workaround, add
a timeout (like we have for the full balance and others) when there are
more devices passed on the command line. This can be skipped by the
--force parameter.
Issue: #708
Signed-off-by: David Sterba <dsterba@suse.com>
strtoull may return the boundary values, if the callers could expect
that and verify it then the errno must be reset before the call.
Signed-off-by: David Sterba <dsterba@suse.com>
