Autoheader uses the AC_DEFINE macros (and a few others) to populate
the config.h.in file. The autotools documentation does not tell
what happens if AC_DEFINE is used twice for the same identifier.
This patch prevents using AC_DEFINE twice for
HAVE_OWN_FIEMAP_EXTENT_DEFINE, preserving the logic (using the
fact that an undefined identifier in a preprocessor directive is
taken as zero).
Signed-off-by: Pierre Labastie <pierre.labastie@neuf.fr>
Signed-off-by: David Sterba <dsterba@suse.com>
Make output of 'btrfs filesystem resize' command more readable and
describe the changes in more detail.
Before:
Resize '/mnt' of '1:-1G'
After:
Resize device id 1 (/dev/vdb) from 4.00GiB to 3.00GiB
Issue: #307
Signed-off-by: Sidong Yang <realwakka@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
To properly check the 64bit timestamp conversion, the filesystem must
support it. Check that a freshly created filesystem contains the
feature.
Signed-off-by: David Sterba <dsterba@suse.com>
Commit b3df561fbf ("btrfs-progs: convert: copy extra timespec on
ext4") has introduced the ability to convert extended inode time
precision on ext4, but this breaks builds on older distros, where ext4
does not have the nsec time precision.
Commit c615287cc0 ("btrfs-progs: a bunch of typo fixes") tried to fix
that by testing the availability of the EXT4_EPOCH_MASK macro, but the
test is not complete.
This patch aims at fixing the macro test, and changes the
name of the associated HAVE_ macro, since the logic is reverted.
This fixes#353 when ext4 has nsec time precision. Note that the test
convert/019-ext4-copy-timestamps fails when ext4 does not have the nsec
time precision and needs to check for the support.
Issue: #353
Signed-off-by: Pierre Labastie <pierre.labastie@neuf.fr>
Signed-off-by: David Sterba <dsterba@suse.com>
The warning is printed for profiles where it's not intended (like raid0
or raid1c4). Check the correct variable for the target profiles.
Issue: #355
Fixes: 1ed5db8db4 ("btrfs-progs: balance convert: add a warning and countdown for RAID56 conversion")
Signed-off-by: David Sterba <dsterba@suse.com>
Image of ext4 with needs_recovery incompat bit set. This bit cannot be
set by regular tune2fs so was created on an empty 4M image by patched
tune2fs that set the bit unconditionally (the image still passed e2fsck,
with journal recovery).
Issue: #348
Signed-off-by: David Sterba <dsterba@suse.com>
As Chris reports: This ext4 file system has 'needs_recovery' feature set, and
if mounted rw, log replay happens. But btrfs-convert doesn't check for it and
converts anyway. It probably shouldn't.
# debugfs -R stats /dev/loop0
debugfs 1.45.6 (20-Mar-2020)
Filesystem volume name: <none>
Last mounted on: /mnt/0
Filesystem UUID: d3e3862e-f892-4ab7-ae91-84eb4be4a3ef
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal ext_attr resize_inode dir_index
filetype needs_recovery extent 64bit flex_bg
sparse_super large_file huge_file dir_nlink
extra_isize metadata_csum
Filesystem flags: signed_directory_hash
Default mount options: user_xattr acl
Filesystem state: clean
Errors behavior: Continue
...
Then 'btrfs-convert' proceeds, while 'e2fsck -fvn /dev/loop1' finds some
problems and wants to fix them.
Add a check for the 'needs_recovery' incompat bit set and don't convert
the filesystem.
Issue: #348
Signed-off-by: David Sterba <dsterba@suse.com>
Hard-coding the pkg-config executable might result in build errors
on system and cross environments that have prefixed toolchains. The
PKG_CONFIG variable already holds the proper one and is already used
in a few other places.
Reviewed-by: Neal Gompa <ngompa13@gmail.com>
Signed-off-by: Heiko Becker <heirecka@exherbo.org>
Signed-off-by: David Sterba <dsterba@suse.com>
When running the tests as root SUDO_HELPER is empty, so that
`run_check "" dd ...` is run, and it fails because the empty command is
not found.
Pull-request: #351
Issue: #352
Author: Pierre Labastie <pierre.labastie@neuf.fr>
Signed-off-by: David Sterba <dsterba@suse.com>
Patch "btrfs-progs: fix false alert on tree block crossing 64K page
boundary" fixes a false alert (warning) when 'btrfs check' is called
right after mkfs.
As we have an extensive mkfs coverage in test mkfs/001, add the check for
the warning there instead of a separate test.
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
When btrfs-check is executed on even newly created fs, it can report
tree blocks crossing 64K page boundary like this:
Opening filesystem to check...
Checking filesystem on /dev/test/test
UUID: 80d734c8-dcbc-411b-9623-a10bd9e7767f
[1/7] checking root items
[2/7] checking extents
WARNING: tree block [30523392, 30539776) crosses 64K page boudnary, may cause problem for 64K page system
[3/7] checking free space cache
[4/7] checking fs roots
[5/7] checking only csums items (without verifying data)
[6/7] checking root refs
[7/7] checking quota groups skipped (not enabled on this FS)
found 131072 bytes used, no error found
total csum bytes: 0
total tree bytes: 131072
total fs tree bytes: 32768
total extent tree bytes: 16384
btree space waste bytes: 125199
file data blocks allocated: 0
referenced 0
[CAUSE]
Tree block [30523392, 30539776) is at the last 16K slot of page.
As 30523392 % 65536 = 49152, and 30539776 % 65536 = 0.
The cross boundary check is using exclusive end, which causes false
alerts.
[FIX]
Use inclusive end to do the cross 64K boundary check.
Reported-by: Wang Yugui <wangyugui@e16-tech.com>
Issue: #352
Issue: #354
Fixes: fc38ae7f48 ("btrfs-progs: check: detect and warn about tree blocks crossing 64K page boundary")
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Currently only single checksum byte is printed. Fix it so that
the whole checksum is printed, in the order as the bytes are stored in
the buffer. This matches what kernel does, though it might not
correspond to the cases of CRC32C and XXHASH as if they were stored in
integer variable and printed in the native format. For consistency we
need to print the same format.
Signed-off-by: Dāvis Mosāns <davispuh@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Enhance --force to also skip the timeout, similar to what --full-balance
does. As this is only to warn about RAID56 that won't be necessary in
the future, don't add a separate option. The warning is still printed.
Signed-off-by: David Sterba <dsterba@suse.com>
Similar to the mkfs warning, add a warning to btrfs balance convert
options, with a countdown to allow the user to have time to cancel the
operation.
Issue: #265
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We all know there's some dark and scary corners with RAID5/6, but users
may not know. Add a warning message in mkfs so anybody trying to use
this will know things can go very wrong.
Issue: #265
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
[ reword message ]
Signed-off-by: David Sterba <dsterba@suse.com>
Some tests report that decompressing the image failed, which did not
fail the test but could lead to wrong errors in case the image is not
overwritten and leaves some old state. Use --force parameter.
[TEST] fuzz-tests.sh
[TEST/fuzz] 001-simple-check-unmounted
xz: btrfs-progs/tests/fuzz-tests/images/bko-97021-invalid-chunk-sectorsize.raw: File exists
failed to decompress image btrfs-progs/tests/fuzz-tests/images/bko-97021-invalid-chunk-sectorsize.raw.xz
[TEST/fuzz] 002-simple-image
xz: btrfs-progs/tests/fuzz-tests/images/bko-97021-invalid-chunk-sectorsize.raw: File exists
failed to decompress image btrfs-progs/tests/fuzz-tests/images/bko-97021-invalid-chunk-sectorsize.raw.xz
Signed-off-by: David Sterba <dsterba@suse.com>
Add scripts that can be used to build docker images and executed from
inside docker containers to verify build or run the testsuite.
Some tweaks are needed at each step to make things work.
- docker-build - build the image
- docker-run - run the default command (test-build)
- run-tests - run the testsuite
Signed-off-by: David Sterba <dsterba@suse.com>
The ci/test-build script unconditionally downloads the latest devel
snapshot. This is not practical for local development. Add a conditional
check for a file named devel.tar.gz, either it's missing or empty, then
download.
The empty file is also considered because this allows to use a docker
image that does not support conditional contents, so a stub file is a
fallback.
Signed-off-by: David Sterba <dsterba@suse.com>
In 5.10 the convert gained support for extended inode time precision,
but this is not available on older distros and breaks build. Add a
configure-time check for the EXT4_EPOCH_MASK macro and add a stub in
case it's not detected.
This means that the 64bit timestamps will not be transferred from the
original filesystem in such environment, at least a warning is printed.
Issue: #344
Signed-off-by: David Sterba <dsterba@suse.com>
Line continuations and not simple "\n" for the json output, this got
inherited to the plain text output, but this is not necessary.
This also caused problems in fstests btrfs/006 where the extra newline
does not match the golden output and the test fails, when printing
device stats that now use the output formatter.
Change the plain text formatting to always expect that a fmt_print or a
manual line print (like is for the device stats) will append the newline
and remove it from the end of formatting.
Link: https://lore.kernel.org/linux-btrfs/CAL3q7H4b7QhL02aSOpN0-k_9P2EAbj1t+NkA6VwidKEg4S996w@mail.gmail.com
Reported-by: Filipe Manana <fdmanana@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This aligns the man page with the usage output of the tool, the notation
in the help text could be confusing as it reads like -r and -i are
mutually exclusive.
Signed-off-by: Christian Amsüss <chrysn@fsfe.org>
Signed-off-by: David Sterba <dsterba@suse.com>
When replace starts with no-background and fails for the reason that
a BTRFS_FS_EXCL_OP is in progress, we still return the value 0 and also
leak the target device open, because in cmd_replace_start() we missed
the goto leave_with_error for this error.
So the test case btrfs/064 in its seqres.full output reports...
Replacing /dev/sdf with /dev/sdc
ERROR: /dev/sdc is mounted
instead of...
Replacing /dev/sdc with /dev/sdf
ERROR: ioctl(DEV_REPLACE_START) '/mnt/scratch': add/delete/balance/replace/resize operation in progress
for the failed replace attempts in the test case
Fix it by jumping to the error label which also fixes the leaked open
device.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The current mount detection code in btrfs receive is not quite perfect.
For example, suppose /tmp is mounted as a tmpfs. In that case,
btrfs receive /tmp2 will find /tmp as the longest mount that matches a
prefix of /tmp2 and blow up because it is not a btrfs filesystem, even
if /tmp2 is just a directory in / mounted as btrfs.
Fix this by replacing the substring check with a dirname recursion to
only check the directories in the path of the dir, rather than every
substring.
Add a new test for this case.
Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>
Add a new subcommand 'btrfs rescue create-control-device' that creates
/dev/btrfs-control. This is helpful on systems that may not have `mknod`
installed and the device node is missing for some reason.
Issue: #223
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Daniel Xu <dxu@dxuuu.xyz>
[ update docs ]
Signed-off-by: David Sterba <dsterba@suse.com>
Introduce a new function, check_test_results(), for
misc/fsck/convert/mkfs test cases.
This function is currently to catch warning message for subpage support,
but can be later expanded for other usages.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
For the incoming subpage support, there is a new requirement for tree
blocks. Tree blocks should not cross 64K page boundary.
For current btrfs-progs and kernel, there shouldn't be any causes to
create such tree blocks. But still, we want to detect such tree blocks
in the wild before subpage support fully lands in upstream.
This patch will add such check for both lowmem and original mode.
Currently it's just a warning, since there aren't many users using 64K
page size yet.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
When the btrfs_read_fs_root() function is searching a ROOT_ITEM with
location key offset other than -1, it currently fails via BUG_ON.
The offset can have other value than -1, though. This can happen for
example if a subvolume is renamed:
$ btrfs subvolume create X && sync
Create subvolume './X'
$ btrfs inspect-internal dump-tree /dev/root | grep -B 2 'name: X$
location key (270 ROOT_ITEM 18446744073709551615) type DIR
transid 283 data_len 0 name_len 1
name: X
$ mv X Y && sync
$ btrfs inspect-internal dump-tree /dev/root | grep -B 2 'name: Y$
location key (270 ROOT_ITEM 0) type DIR
transid 285 data_len 0 name_len 1
name: Y
As can be seen the offset changed from -1ULL to 0.
Do not fail in this case.
Signed-off-by: Marek Behún <marek.behun@nic.cz>
CC: Qu Wenruo <wqu@suse.com>
CC: Tom Rini <trini@konsulko.com>
Signed-off-by: David Sterba <dsterba@suse.com>
btrfs_open_dir already has a check whether the passed path is a
directory and if so it returns a specific error code (-3) when such an
error occurs. Use this instead of open-coding the directory check. To
avoid regression in cli/003 test also move directory checks before fs
type in btrfs_open.
Output before this check:
ERROR: resize works on mounted filesystems and accepts only
directories as argument. Passing file containing a btrfs image
would resize the underlying filesystem instead of the image.
After:
ERROR: not a directory: /root/btrfs-progs/tests/test.img
ERROR: resize works on mounted filesystems and accepts only
directories as argument. Passing file containing a btrfs image
would resize the underlying filesystem instead of the image.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add a test case which ensures that when resize is tried on an image
instead of a directory appropriate warning is produced and the command
fails.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This reverts commit 61ecaff036.
The libmount functionality is not used anymore, we can remove it
entirely.
Signed-off-by: David Sterba <dsterba@suse.com>
Partial revert of 922eaa7b54 ("btrfs-progs: build: fix linking with
static libmount"), remove the necessary workarounds like the weak
symbols and link time warnings. Symbols renamed not to clash with
libmount (parse_size, canonicalize_path) haven't been reverted because
the new names are acceptable.
Signed-off-by: David Sterba <dsterba@suse.com>
This reverts commit 98a88aec64.
The libmount dependency will be dropped so remove it from the
documentation as well.
Signed-off-by: David Sterba <dsterba@suse.com>
In commit 57cfe29e69 ("btrfs-progs: utils: introduce
find_mount_fsroot") the entries in /proc/self/mountinfo are parsed by a
convenience library libmount, because getmntent does not provide the
information we need to distinguish bind mounts.
Using libmount turned out to be problematic in several ways:
- static build got broken due to clashing symbols, eg. for parsing size
or path canonicalization (#333)
- long-term distros do not have libmount new enough (2.24+) to provide
some functions (mnt_table_is_empty, #334)
- libmount internally uses getgrnam_r/mnt_get_uid/... that are not
static-build friendly, a warning is printed during link time for each
binary; we don't use any of the functions
- libmount has further library dependencies that we don't need:
$ ldd /usr/lib64/libmount.so.1
linux-vdso.so.1 (0x00007fff4f175000)
libc.so.6 => /lib64/libc.so.6 (0x00007f44a1763000)
libblkid.so.1 => /usr/lib64/libblkid.so.1 (0x00007f44a1730000)
libselinux.so.1 => /usr/lib64/libselinux.so.1 (0x00007f44a1704000)
/lib64/ld-linux-x86-64.so.2 (0x00007f44a1998000)
libpcre.so.1 => /usr/lib64/libpcre.so.1 (0x00007f44a166c000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f44a1666000)
namely selinux, pcre and dl.
Summing it up, libmount causes more trouble than it's worth using a
convenience library, we want to keep the dependencies minimal so the
custom mountinfo parser was inevitable.
Issue: #333
Issue: #334
Issue: #336
Signed-off-by: David Sterba <dsterba@suse.com>
We had a few bugs on the kernel side of send/receive where capabilities
ended up being lost after receiving a send stream. They all stem from the
fact that the kernel used to send all xattrs before issuing the chown
command, and the later clears any existing capabilities in a file or
directory.
Initially a workaround was added to btrfs-progs' receive command, in commit
123a2a0850 ("btrfs-progs: receive: restore capabilities after chown"),
and that fixed some instances of the problem. More recently, other instances
of the problem were found, a proper fix for the kernel was made, which fixes
the root problem by making send always emit the setxattr command for setting
capabilities after issuing a chown command. This was done in kernel commit
89efda52e6b693 ("btrfs: send: emit file capabilities after chown"), which
landed in kernel 5.8.
However, the workaround on the receive command now causes us to incorrectly
set a capability on a file that should not have it, because it assumes all
setxattr commands for a file always comes before a chown.
Example reproducer:
$ cat send-caps.sh
#!/bin/bash
DEV1=/dev/sdh
DEV2=/dev/sdi
MNT1=/mnt/sdh
MNT2=/mnt/sdi
mkfs.btrfs -f $DEV1 > /dev/null
mkfs.btrfs -f $DEV2 > /dev/null
mount $DEV1 $MNT1
mount $DEV2 $MNT2
touch $MNT1/foo
touch $MNT1/bar
setcap cap_net_raw=p $MNT1/foo
btrfs subvolume snapshot -r $MNT1 $MNT1/snap1
btrfs send $MNT1/snap1 | btrfs receive $MNT2
echo
echo "capabilities on destination filesystem:"
echo
getcap $MNT2/snap1/foo
getcap $MNT2/snap1/bar
umount $MNT1
umount $MNT2
When running the test script, we can see that both files foo and bar get
the capability set, when only file foo should have it:
$ ./send-caps.sh
Create a readonly snapshot of '/mnt/sdh' in '/mnt/sdh/snap1'
At subvol /mnt/sdh/snap1
At subvol snap1
capabilities on destination filesystem:
/mnt/sdi/snap1/foo cap_net_raw=p
/mnt/sdi/snap1/bar cap_net_raw=p
Since the kernel fix was backported to all currently supported stable
releases (5.10.x, 5.4.x, 4.19.x, 4.14.x, 4.9.x and 4.4.x), remove the
workaround from receive. Having such a workaround relying on the order
of commands in a send stream is always troublesome and doomed to break
one day.
A test case for fstests will come soon.
Issue: #85
Issue: #202
Issue: #292
Reported-by: Richard Brown <rbrown@suse.de>
Reviewed-by: Su Yue <l@damenly.su>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
