Free memory after errors in sum(), this is reported by gcc 13 on the CI.
This was reproduced by:
$ make D=asan TEST=019-receive-clones-on-mounted-subvol test-misc
Signed-off-by: David Sterba <dsterba@suse.com>
Test the following:
- Create a btrfs with crc32c csum
- Populate the filesystem
- Convert the filesystem to the following csums:
* xxhash
* blake2
* sha256
* crc32c
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This test case will check the following behviour:
- Regular zoned supported data/metadata profiles
Should success
- RST with zoned feature
Should fail for non-experimental builds
- zoned with data profiles that only RST supports
Should fail for non-experimental builds.
Meanwhile for experimental builds it should success and enable RST
feature automatically (verified in mkfs/030)
Pull-request: #813
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Currently we unconditionally run mkfs/029 and mkfs/030 as we export
RST feature through mkfs.btrfs as supported features.
But considering the mkfs.btrfs features don't match kernel support (only
a BTRFS_DEBUG kernel can support that), we're going to hide RST behind
experimental builds.
In that case RST related tests cases also need to be behind experimental
builds as regular builds of mkfs.btrfs will no longer support RST
feature.
Introduce two helpers:
- _test_config()
To verify if certain config is set to 1
- check_experimental_build()
A wrapper of "_test_config EXPERIMENTAL", and skip the test if
btrfs-progs has no experimental features.
So that test cases can be skipped for non-experimental builds.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The raid-stripe-tree (RST) feature is for zoned devices to support extra
data profiles, and is not yet a stable feature (still requires
CONFIG_BTRFS_DEBUG enabled kernel to support it).
Furthermore the supported filesystems (ext*, reiserfs and ntfs) don't
even support zoned devices, and even we force RST support for
btrfs-convert, we would only create an empty tree for RST, as btrfs
convert would only result SINGLE data profile with SINGLE/DUP metadata
profile, neither needs RST at all.
Enabling RST for btrfs-convert would only cause problems for false test
failures as we incorrectly allow RST feature for btrfs-convert.
Fixes the problem by removing raid-stripe-tree support from
btrfs-convert and remove the test cases support for RST.
This patch is mostly reverting commit 346a381923 ("btrfs-progs:
convert: add raid-stripe-tree to allowed features"), but keeps the test
infrastructure to support bgt features for convert.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
There is a random failure for misc/028 in github CI, but unable to
reproduce locally.
[EXTRA DUMP]
The failure is at mount time, which is mostly out of our control (kernel
is provided by the CI image), but at least we can do extra kernel dump
if a mount fails.
This dump is done inside run_check() where if we detects there is a
"mount" word inside the command string.
The detection is not the best, it may detect other commands that
contains the word "mount", but it should be good enough to detect real
mount failure.
Pull-request: #815
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
There is a random chance that misc/055 would fail with stale qgroups
detected.
[CAUSE]
Commit 82f7d6c1d7 ("btrfs-progs: qgroup: handle stale qgroup deletion
more accurately") changed the behavior of "btrfs qgroup clear-stale"
that it will no longer try to remove qgroup under deletion.
And the test case itself relies on the return value of "btrfs qgroup
clear-stale" to do the extra wait for subvolume deletion.
This means after that commit, the test case would skip the wait if there
is a subvolume waiting for cleanup, resulting in a race window where the
subvolume can be dropped and become stale, and eventually trigger the
stale qgroup detection and cause false alerts.
[FIX]
Fix the test case by always wait for the subvolume to be dropped, so
that later "btrfs qgroup clear-stale" can properly remove all stale
qgroups.
Pull-request: #814
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The test check/037-freespacetree-repair sometimes fails (does not in the
CI though) that the last unmount does not work due to -EBUSY. This
depends on timing, but what is obviously wrong is the 'rm' before
umount. The quoting of "*" does not remove all the files before
fallocate but is otherwise silent due to '-f'.
Add the run_check so it's recorded in the logs taht it happens and alos
add sudo so it can actually work. This makes the test work more
reliably.
Signed-off-by: David Sterba <dsterba@suse.com>
Fix the following issues in the test suite:
- lack of quoting for variables
- declare function variables local when missing (prevent accidental
overwrite of global variables)
- for variables with underscore in the name use plain "$VAR_NAME"
instead of { } (unless necessary)
- minor style adjustments like moving quotes to the end of the same
string
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
There is a long existing failure in my local VM that with any newer
kernel (6.x) the test case misc/004 would fail with ENOSPC during
balance:
[TEST] misc-tests.sh
[TEST/misc] 004-shrink-fs
failed: /home/adam/btrfs-progs/btrfs balance start -mconvert=single -sconvert=single -f /home/adam/btrfs-progs/tests/mnt
test failed for case 004-shrink-fs
make: *** [Makefile:547: test-misc] Error 1
[CAUSE]
With more testing, it turns out that just before the balance, the
filesystem still have several empty data block groups.
The reason is the new default discard=async behavior, as it also changes
the empty block groups to be async, this leave the empty block groups
there, resulting no extra space for the convert balance.
[FIX]
I do not understand why for loop block devices we also enable
discard, but at least disable discard for the test case so that we can
ensure the empty block groups get cleaned up properly.
Pull-request: #809
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The test case always fails in my VM, with the following error:
$ sudo TEST=038\* make test-misc
[TEST] misc-tests.sh
[TEST/misc] 038-backup-root-corruption
Backup 2 not overwritten
test failed for case 038-backup-root-corruption
After more debugging, it turns out that there is nothing wrong except
the final check:
[ "$main_root_ptr" -ne "$backup_new_root_ptr" ] || _fail "Backup 2 not overwritten"
The _fail() is only triggered if the previous check returns false, which
is completely the opposite.
Furthermore on the github CI, the kernel commits 2 instead of 1
transaction, resulting the next slot never to match the current
generation/tree root.
The two bugs combined, github CI always passses the test case, while
for my VM which does the expected one transaction, it would always fail.
Fix it by:
- Use a proper "if [] then; fi" block to check the tree root bytenr
- Use the generation diff to calculate the expected backup root slot
- Log the full super block dump for debug usage
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Executing the script inside the directories as './test.sh' is not
supposed to work but could happen accidentally. With an exit after
attempting to source the we can fix that. Not all cases have been fixed
in f6bbe06c08 ("btrfs-progs: tests: add protection against running
out of test suite").
Signed-off-by: David Sterba <dsterba@suse.com>
The test case verifies behavior of ext4 unwritten extents:
- Create a unwritten (preallocated) extent on ext4
- Fill the on-disk extent with random garbage
This is to make sure if btrfs tries to read the on-disk data, it would
definitely get some garbage.
As I found sometimes mkfs.ext4 can fill the unused bg with zeros.
- Fill the preallocated file range with some data
This is to make sure btrfs-convert can handle mixed written and
unwritten ranges.
- Save the checksum of the file.
- Convert the fs
- Verify the checksum
For older btrfs-convert, there would be only one regular file extent,
and reading the file would read out some garbage and cause checksum to
mismatch.
For the fixed btrfs-convert, we punch holes for unwritten extents,
thus only the written part would be read out and match the checksum.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
With the change of minimal number of zones, mkfs-tests/030-zoned-rst now
fails because the loopback device is 2GB and can contain 8x 256MB zones.
Use the nullb helpers to choose a smaller zone size.
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add test for mkfs.btrfs zone reset behavior to check if
- it resets all the zones without "-b" option
- it detects an active zone outside of the FS range
- it do not reset a zone outside of the range
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
After patch "btrfs-progs: qgroup: handle stale qgroup deletion more
accurately" cleaning stale qgroups may not happen if the subvolume
cleaning is still in progress. Update the test so it's able to handle
that.
Signed-off-by: David Sterba <dsterba@suse.com>
The new test case would contain a file system image, with the following
file:
item 4 key (257 INODE_ITEM 0) itemoff 15883 itemsize 160
generation 7 transid 8 size 16384 nbytes 16384
block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
sequence 258 flags 0x0(none)
atime 1714635006.328482575 (2024-05-02 17:00:06)
ctime 1714635013.394980640 (2024-05-02 17:00:13)
mtime 1714635013.394980640 (2024-05-02 17:00:13)
otime 1714635006.328482575 (2024-05-02 17:00:06)
item 5 key (257 INODE_REF 256) itemoff 15869 itemsize 14
index 2 namelen 4 name: file
item 6 key (257 EXTENT_DATA 0) itemoff 15816 itemsize 53
generation 7 type 1 (regular)
extent data disk byte 13631488 nr 1048576
extent data offset 0 nr 16384 ram 16384
extent compression 0 (none)
Note the ram bytes, which should be 1048576.
Furthermore, the inode size is truncated to 16K (originally 1M), so that
offset + num_bytes would still be no larger than ram_bytes.
So the only error is the mismatch between ram_bytes and disk_num_bytes
for the non-compressed data extent.
The image is hand crafted for now, as btrfs-corrupt-block doesn't not yet
support corrupting the ram_bytes field.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The test case relies on `--delete-qgroup` option which has been removed.
The feature needs to be implemented in kernel as it's more complex than
just calling an ioctl. The test case does not take the complexity of
subvolume dropping into consideration and only tested the simplest
cases.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add a new test case to make sure:
- btrfstune can convert a zoned btrfs with extent tree to bgt
- btrfstune can convert a zoned btrfs with bgt back to extent tree
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Reviewed-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Bit shifts should be done on unsigned type as a matter of good practice
to avoid any problems with bit overflowing to the sign bit.
Signed-off-by: David Sterba <dsterba@suse.com>
There are now two tests with 060, we'd like to be that a unique number,
rename the one that was added more recently.
Signed-off-by: David Sterba <dsterba@suse.com>
The raid-stripe-tree can be enabled for convert, though it's still
considered incomplete and slightly experimental. Due to that the tests
need to be adjusted to check for support and skip mount eventually.
Possible remaining options to add: quota, squota
Issue: #694
Signed-off-by: David Sterba <dsterba@suse.com>
Enhance the conversion macro to recognize the filesystem type in case
the ntfs-specific converter utility has to be used. Its current feature
set does not match btrfs-convert.
Signed-off-by: David Sterba <dsterba@suse.com>
The test does not pass in environments where /bin/sh is not bash (like
github CI). Unify the initial setup like the other tests do: source,
compatibility, prerequisities, the rest.
Signed-off-by: David Sterba <dsterba@suse.com>
The strict check is enabled for both check_image() and
convert_test_do_convert() to detect chunk alignment related problems.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The test case would check if "btrfs subvolume create":
- Report error on an existing path
- Still report error if mulitple paths are given and one of them already
exists
- For above case, still created a subvolume for the good parameter
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Since we're already directing the end user to use "btrfs rescue
clear-ino-cache" command, there is not much need to support it in
btrfs-check.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
https://www.openssl.org/ Is a well known cryptography library and since
freshly released version 3.2 it also supports variable digest size of
blake2b, so we can now add it among the crypto providers.
Configure with --with-crypto=openssl.
Signed-off-by: David Sterba <dsterba@suse.com>
https://botan.randombit.net/ Botan is a cryptography library with C
bindings and provides what we need (sha256 and blake2b), among many
others. Add it to the list of crypto backends if somebody wants to use
it.
Currently the version 2.19 is the latest one. Botan3 3.2.0 exists but
does not seem to be widely available in distros yet.
Configure with --with-crypto=botan.
Signed-off-by: David Sterba <dsterba@suse.com>
The test case would verify the following behaviors:
- Partial failure
Should return 1, but the remaining valid destinations would still be
created.
- All success
That's as usual.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
With the kernel commit 070bb0011ccf ("btrfs: sysfs: show if ACL
support has been compiled in") we can now check if ACL is compiled
without requiring a btrfs device. Retain older method for older
kernels.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The zone size may not be parsed correctly in 'nullb create' due to
copy&paste error from the 'size'. This is already fixed upstream.
Signed-off-by: David Sterba <dsterba@suse.com>
Since a few days the CI started to fail randomly when there were loop
devices used in the tests. The mount fails because some device is
reported to be missing:
$ losetup --show --find
/dev/loop3
...
$ mkfs ...
ERROR: device scan failed on '/dev/loop3': No such file or directory
...
$ mount
mount: /home/runner/work/btrfs-progs/btrfs-progs/tests/mnt: wrong fs
type, bad option, bad superblock on /dev/loop3, missing codepage or
helper program, or other error.
$ dmesg
...
BTRFS error (device loop0): devid 3 uuid 11d9c345-9527-433e-a024-7102659fa0ee is missing
BTRFS error (device loop0): failed to read the system array: -2
BTRFS error (device loop0): open_ctree failed
This was reproducible in the "cli" tests, but also happened on a local
machine.
To fix that wait for all loop devices before mount, the command
'btrfs device ready' should block until that. The convenience helper
does that, for any standalone 'mount' used with loop devices this must
be done manually.
Signed-off-by: David Sterba <dsterba@suse.com>
The test names 011 and 012 are the same but 012 is actually testing
recursion of the 'btrfs fi du' command.
Signed-off-by: David Sterba <dsterba@suse.com>
There was a support for a short syntax of 'btrfs balance' that accepted
a path where normally would be the mandatory subcommand. This was a
heuristic and nowadays everybody should be using the
'btrfs balance action' syntax. The warning was in place for a year, it's
time to remove the short syntax completely.
Issue: #517
Signed-off-by: David Sterba <dsterba@suse.com>
The most common sector sizes are 4k, 16k and 64k, we don't really need
to test that with convert, this takes a long time for little benefit as
the node size is only for metadata, while the rest remains the same.
Signed-off-by: David Sterba <dsterba@suse.com>
