This is in keeping with what the function actually does, and is named
this way in the kernel.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This is a bit larger than the previous syncs, because we use
extent_io_tree's everywhere. There's a lot of stuff added to
kerncompat.h, and then I went through and cleaned up all the API
changes, which were
- extent_io_tree_init takes an fs_info and an owner now.
- extent_io_tree_cleanup is now extent_io_tree_release.
- set_extent_dirty takes a gfpmask.
- clear_extent_dirty takes a cached_state.
- find_first_extent_bit takes a cached_state.
The diffstat looks insane for this, but keep in mind extent-io-tree.c
and extent-io-tree.h are ~2000 loc just by themselves.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This patch syncs file-item.h into btrfs-progs. This carries with it an
API change for btrfs_del_csums, which takes a root argument in the
kernel, so all callsites have been updated accordingly.
I didn't sync file-item.c because it carries with it a bunch of bio
related helpers which are difficult to adapt to the kernel.
Additionally there's a few helpers in the local copy of file-item.c that
aren't in the kernel that are required for different tools.
This requires more cleanups in both the kernel and progs in order to
sync file-item.c, so for now just do file-item.h in order to pull things
out of ctree.h.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This syncs accessors.[ch] from the kernel. For the most part
accessors.h will remain the same, there's just some helpers that need to
be adjusted for eb->data instead of eb->pages. Additionally accessors.c
needed to be completely updated to deal with this as well.
This is a set of files where we will likely only sync the header going
forward, and leave the C file in place as it needs to be specific to
btrfs-progs.
This forced a few "unrelated" changes
- Using btrfs_dir_item_ftype() instead of btrfs_dir_item_type(). This
is due to the encryption changes, and was simpler to just do in this
patch.
- Adjusting some of the print tree code to use the actual helpers and
not the btrfs-progs ones.
A local definition of static_assert is used to avoid compilation
failures on older gcc (< 9) where the 2nd parameter is mandatory.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We want to keep this file locally as we want to be uptodate with
upstream, so we can build btrfs-progs regardless of which kernel is
currently installed. Sync this with the upstream version and put it in
kernel-shared/uapi to maintain some semblance of where this file comes
from.
There are some changes that need to be synced back to kernel. A local
definition of static_assert is used to avoid compilation problems on gcc
(< 9) due to mandatory 2nd parameter.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Now that the libbtrfs stuff has it's own local copy of ctree.h and
ioctl.h, let's rename these qgroup struct members to match the kernel
names, this way it'll make it easier to sync the kernel code into
btrfs-progs.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
While syncing messages.[ch] I had to back out the ASSERT() code in
kerncompat.h, which means we now rely on the kernel code for ASSERT().
In order to maintain some semblance of separation introduce UASSERT()
and use that in all the purely userspace code.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Running 'btrfs inspect tree-stats' on a mounted filesystem works though
it reads directly from block devices. This can lead to inconsistent
data, warnings, errors or or a crash. More checks could be added but at
least explain things in more detail.
Issue: #520
Signed-off-by: David Sterba <dsterba@suse.com>
Attempting to create a snapshot of subvolume with an active swapfile
prints the errno message corresponding to ETXTBSY but this is confusing
so change it to be more descriptove to:
ERROR: cannot snapshot '/hibernate': source subvolume contains an active swapfile (Text file busy)
Issue: #607
Signed-off-by: David Sterba <dsterba@suse.com>
The *64 interfaces, such as fstat64, off64_t, etc, are legacy interfaces
created at a time when 64-bit file support was still new. They are
generally exposed when defining a macro named _LARGEFILE64_SOURCE, as
e.g. the glibc docs[0] say.
The modern way to utilise largefile support, is to continue to use the
regular interfaces (off_t, fstat, ..), and define _FILE_OFFSET_BITS=64.
We already use the autoconf macro AC_SYS_LARGEFILE[1] which arranges this
and sets this macro for us. Therefore, we can utilise the non-64 names
without fear of breaking on 32-bit systems.
This fixes the build against musl libc, ever since musl dropped the
*64 compat from interfaces by default[2] just for _GNU_SOURCE, unless
_LARGEFILE64_SOURCE is defined. However, there are plans for a future
removal of the whole *64 header API, and that workaround (adding another
define) might cease to exist.
So, rename all *64 API use to the regular non-suffixed names. For
consistency, rename the internal functions that were *64 named
(lstat64_path, ..) too.
This should have no regressions on any platform.
[0]: https://www.gnu.org/software/libc/manual/html_node/Feature-Test-Macros.html#index-_005fLARGEFILE64_005fSOURCE
[1]: https://www.gnu.org/software/autoconf/manual/autoconf-2.67/html_node/System-Services.html
[2]: 25e6fee27f
Pull-request: #615
Signed-off-by: psykose <alice@ayaya.dev>
Signed-off-by: David Sterba <dsterba@suse.com>
The commit 6f7151f499 extended the set of recognized valid subcommands
for the old path syntax but wrongly checks for more than 2 parameters.
That way a shortened and valid new syntax is not recognized (here 'can'
is short for 'cancel' and the short form is not in the list):
btrfs-progs-6.1.3
btrfs bal can /
ERROR: balance cancel on '/' failed: Not in progress
btrfs-progs-6.2.2
btrfs bal can /
WARNING: deprecated syntax, please use 'btrfs balance start'
ERROR: cannot access 'can': No such file or directory
Issue: #612
Fixes: 6f7151f499 ("btrfs-progs: balance: fix some cases wrongly parsed as old syntax")
Signed-off-by: David Sterba <dsterba@suse.com>
For older kernels, the sysfs interface providing the fsid may not be present yet.
Since 32c2e57c65 ("btrfs-progs: read fsid from the sysfs in
device_is_seed"), the fallback to the previous approach to determine
the fsid was not used anymore.
Ensure negative return values of sysfs_open_fsid_file are handled by
falling back to the dev_to_fsid in this case.
Pull-request: #599
Signed-off-by: David Sterba <dsterba@suse.com>
Deprecate old 'btrfs balance' syntax since new syntax has been
introduced in 2012. We will remove the old syntax completely in a few
releases.
Signed-off-by: Wang Yugui <wangyugui@e16-tech.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Some cases of 'btrfs balance' are wrongly parsed as old syntax.
$ btrfs balance status
ERROR: cannot access 'status': No such file or directory
Currently, only 'start' is successfully excluded in the check of old
syntax. Fix it by adding others in the check of old syntax.
Signed-off-by: Wang Yugui <wangyugui@e16-tech.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In Dry run the following error appeared and aborted execution:
ERROR: failed to access 'XYZ' to restore metadata/xattrs: No such file or directory
Skip the metadata and xattrs handling when dry run is enabled.
Signed-off-by: Holger Jakob <jakob@dsi.uni-stuttgart.de>
Signed-off-by: David Sterba <dsterba@suse.com>
Restore was only setting xattrs on files but ignored directories.
Signed-off-by: Holger Jakob <jakob@dsi.uni-stuttgart.de>
Signed-off-by: David Sterba <dsterba@suse.com>
On a 32bit host the split qgroupid is wrong due to the way the numbers
are passed to the formatter as variable length arguments. The level is
u16, promoted to int and then parsed as u64. This means that the values
are shifted and some stack data are printed instead.
Example error messages from yast2-bootloader:
SystemCmd.cc(addLine):569 Adding Line 7 " "qgroupid": "21474836480/23885859321282560","
The value 21474836480 = 0x5000000 is 0x5 shifted by 32 bits,
23885859321282560 is 0x54dc1000000000 and shifting by 32 does not
lead to a valid value which should be 0 in this case.
Bugzilla: https://bugzilla.suse.com/show_bug.cgi?id=1209136
Signed-off-by: David Sterba <dsterba@suse.com>
The tabular output prints the same value for all columns:
# btrfs device stats /srv/btrfs-data
[/dev/sdc1].write_io_errs 0
[/dev/sdc1].read_io_errs 0
[/dev/sdc1].flush_io_errs 0
[/dev/sdc1].corruption_errs 0
[/dev/sdc1].generation_errs 0
[/dev/sdb1].write_io_errs 7489899
[/dev/sdb1].read_io_errs 3751023
[/dev/sdb1].flush_io_errs 117
[/dev/sdb1].corruption_errs 68
[/dev/sdb1].generation_errs 25
# btrfs device stats -T /srv/btrfs-data
Id Path Write errors Read errors Flush errors Corruption errors Generation errors
-- --------- ------------ ----------- ------------ ----------------- -----------------
1 /dev/sdc1 0 0 0 0 0
2 /dev/sdb1 25 25 25 25 25
The table_printf has a fixed list of columns and should not iterate over
them. Only check if some of the value is set and return error.
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=217045
Issue: #585
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
Currently cli/009 test case failed with different exit number:
====== RUN CHECK /home/adam/btrfs-progs/btrfstune --help
usage: btrfstune [options] device
[...]
failed: /home/adam/btrfs-progs/btrfstune --help
test failed for case 009-btrfstune
[CAUSE]
In tune/main.c, we have the following call on usage():
static void print_usage(int ret)
{
usage(&tune_cmd);
exit(ret);
}
However usage() itself would always call exit(1):
void usage(const struct cmd_struct *cmd)
{
usage_command_usagestr(cmd->usagestr, NULL, 0, true, true);
exit(1);
}
This makes prevents any caller of usage() to modify its exit number.
[FIX]
Add a new argument @error for print_usage(), so we can properly return 0
for -h/--help usage.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The kernel commit a26d60dedf9a ("btrfs: sysfs: add devinfo/fsid to
retrieve actual fsid from the device") introduced a sysfs interface
to access the device's fsid from the userspace. This is a more
reliable method to obtain the fsid compared to reading the
superblock, and it even works if the device is not present.
Additionally, this sysfs interface can be read by non-root users.
Therefore, it is recommended to utilize this new sysfs interface to
retrieve the fsid.
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
load_device_info() checks if the device is a seed device by reading
superblock::fsid and comparing it with the mount fsid, and it fails
to work if the device is missing (a RAID1 seed fs). Move this part
of the code into a new helper function device_is_seed() in
preparation to make device_is_seed() work with the new sysfs
devinfo/<devid>/fsid interface.
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add option --uuid with same semantics that is provided by command
'mkswap'. By default a random UUID is generated, to not set any use
'btrfs filesystem mkswapfile -U clear swapfile'.
Issue: #581
Signed-off-by: David Sterba <dsterba@suse.com>
The CI build on musl warns:
[CC] cmds/balance.o
In file included from cmds/inspect.c:19:
/usr/include/sys/fcntl.h:1:2: warning: #warning redirecting incorrect #include <sys/fcntl.h> to <fcntl.h> [-Wcpp]
1 | #warning redirecting incorrect #include <sys/fcntl.h> to <fcntl.h>
| ^~~~~~~
On glibc the header directly includes fcntl.h.
Signed-off-by: David Sterba <dsterba@suse.com>
[WARNING]
Clang 15.0.7 warns about several unused variables:
kernel-shared/zoned.c:829:6: warning: variable 'num_sequential' set but not used [-Wunused-but-set-variable]
u32 num_sequential = 0, num_conventional = 0;
^
cmds/scrub.c:1174:6: warning: variable 'n_skip' set but not used [-Wunused-but-set-variable]
int n_skip = 0;
^
mkfs/main.c:493:6: warning: variable 'total_block_count' set but not used [-Wunused-but-set-variable]
u64 total_block_count = 0;
^
image/main.c:2246:6: warning: variable 'bytenr' set but not used [-Wunused-but-set-variable]
u64 bytenr = 0;
^
[CAUSE]
Most of them are just straightforward set but not used variables.
The only exception is total_block_count, which has commented out code
relying on it.
[FIX]
Just remove those variables, and for @total_block_count, also remove the
comments.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[FALSE ALERT]
Unlike gcc, clang doesn't really understand the comments, thus it's
reportings tons of fall through related errors:
cmds/reflink.c:124:3: warning: unannotated fall-through between switch labels [-Wimplicit-fallthrough]
case 'r':
^
cmds/reflink.c:124:3: note: insert '__attribute__((fallthrough));' to silence this warning
case 'r':
^
__attribute__((fallthrough));
cmds/reflink.c:124:3: note: insert 'break;' to avoid fall-through
case 'r':
^
break;
[CAUSE]
Although gcc is fine with /* fallthrough */ comments, clang is not.
[FIX]
So just introduce a fallthrough macro to handle the situation properly,
and use that macro instead.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Before decompressing, we zero out the content of the entire output buffer,
so that we don't get any garbage after the last byte of data. We do this
for all compression algorithms. However zstd, at least with libzstd 1.5.2
on Debian (version 1.5.2+dfsg-1), the decompression routine can end up
touching the content of the output buffer beyond the last valid byte of
decompressed data, resulting in a corruption.
Example reproducer:
$ cat test.sh
#!/bin/bash
DEV=/dev/sdj
MNT=/mnt/sdj
rm -f /tmp/send.stream
umount $DEV &> /dev/null
mkfs.btrfs -f $DEV &> /dev/null || echo "MKFS failed!"
mount -o compress=zstd $DEV $MNT
# File foo is not sector size aligned, 127K.
xfs_io -f -c "pwrite -S 0xab 0 3" \
-c "pwrite -S 0xcd 3 130042" \
-c "pwrite -S 0xef 130045 3" $MNT/foo
# Now do an fallocate that increases the size of foo from 127K to 128K.
xfs_io -c "falloc 0 128K " $MNT/foo
btrfs subvolume snapshot -r $MNT $MNT/snap
btrfs send --compressed-data -f /tmp/send.stream $MNT/snap
echo -e "\nFile data in the original filesystem:\n"
od -A d -t x1 $MNT/snap/foo
umount $MNT
mkfs.btrfs -f $DEV &> /dev/null || echo "MKFS failed!"
mount $DEV $MNT
btrfs receive --force-decompress -f /tmp/send.stream $MNT
echo -e "\nFile data in the new filesystem:\n"
od -A d -t x1 $MNT/snap/foo
umount $MNT
Running the reproducer gives:
$ ./test.sh
(...)
File data in the original filesystem:
0000000 ab ab ab cd cd cd cd cd cd cd cd cd cd cd cd cd
0000016 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd
*
0130032 cd cd cd cd cd cd cd cd cd cd cd cd cd ef ef ef
0130048 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
0131072
At subvol snap
File data in the new filesystem:
0000000 ab ab ab cd cd cd cd cd cd cd cd cd cd cd cd cd
0000016 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd
*
0130032 cd cd cd cd cd cd cd cd cd cd cd cd cd ef ef ef
0130048 cd cd cd cd 00 00 00 00 00 00 00 00 00 00 00 00
0130064 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
0131072
The are 4 bytes with a value of 0xcd instead of 0x00, at file offset
127K (130048).
Fix this by explicitly zeroing out the part of the output buffer that was
not used after decompressing with zstd.
The decompression of compressed extents, sent when using the send v2
stream, happens in the following cases:
1) By explicitly passing --force-decompress to the receive command, as in
the reproducer above;
2) Calling the BTRFS_IOC_ENCODED_WRITE ioctl failed with -ENOTTY, meaning
the kernel on the receiving side is old and does not implement that
ioctl;
3) Calling the BTRFS_IOC_ENCODED_WRITE ioctl failed with -ENOSPC;
4) Calling the BTRFS_IOC_ENCODED_WRITE ioctl failed with -EINVAL.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
By default print how many subvolumes are considered for checks, either
found or specified on the command line. Once a subvolume is removed from
the list print the progress from the total count.
$ btrfs subvolume sync /path
Waiting for 130 subvolumes
Subvolume id 256 is gone (1/130)
Subvolume id 257 is gone (2/130)
...
Subvolume id 384 is gone (129/130)
Subvolume id 385 is gone (130/130)
Signed-off-by: David Sterba <dsterba@suse.com>
Per user report on https://old.reddit.com/r/btrfs/comments/107fnw1/btrfs_filesystem_mkswapfile_results_in_an/
the swapfile header does not contain the correct number of pages that
matches the file size and the activated swapfile is only 1GiB:
# btrfs filesystem mkswapfile -s 10g swapfile
# swapon swapfile
# cat /proc/swaps
Filename Type Size Used Priority
/swap/swapfile file 1048572 0 -2
A workaround is to run 'mkswap swapfile' before activation. Proper fix
is to calculate the number of (fixed size) 4K pages available for the
swap.
Issue: #568
Signed-off-by: David Sterba <dsterba@suse.com>
In default build there's a warning (reported by CI) that the
experimental list-chunk command and related functions are not used, so
add the condition there as well.
In file included from cmds/inspect.c:45:
./cmds/commands.h:67:26: warning: 'cmd_struct_inspect_list_chunks' defined but not used [-Wunused-const-variable=]
#define __CMD_NAME(name) cmd_struct_ ##name
^~~~~~~~~~~
./cmds/commands.h:73:26: note: in expansion of macro '__CMD_NAME'
const struct cmd_struct __CMD_NAME(name) = \
^~~~~~~~~~
./cmds/commands.h:88:2: note: in expansion of macro 'DEFINE_COMMAND'
DEFINE_COMMAND(name, token, cmd_ ##name, \
^~~~~~~~~~~~~~
cmds/inspect.c:1115:8: note: in expansion of macro 'DEFINE_SIMPLE_COMMAND'
static DEFINE_SIMPLE_COMMAND(inspect_list_chunks, "list-chunks");
^~~~~~~~~~~~~~~~~~~~~
Signed-off-by: David Sterba <dsterba@suse.com>
This reverts commit 03451430de.
(It's not 1:1, there are some additional trivial fixups in cmds/qgroup.c)
This breaks a lot of 3rd party tools that depend on it as Neal reports:
* btrfs-assistant
* buildah
* cri-o
* podman
* skopeo
* containerd
* moby/docker
* snapper
* source-to-image
Link: https://lore.kernel.org/linux-btrfs/CAEg-Je8L7jieKdoWoZBuBZ6RdXwvwrx04AB0fOZF1fr5Pb-o1g@mail.gmail.com/
Reported-by: Neal Gompa <ngompa@fedoraproject.org>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
Since commit d729048be6 ("btrfs-progs: stop using
btrfs_root_item_v0"), "btrfs subvolume list -u" not longer correctly
reports UUID nor timestamp, while older (btrfs-progs v6.0.2) still works
correctly:
v6.0.2:
# btrfs subv list -u /mnt/btrfs/
ID 256 gen 12 top level 5 uuid ed4af580-d512-2644-b392-2a71aaeeb99e path subv1
ID 257 gen 13 top level 5 uuid a22ccba7-0a0a-a94f-af4b-5116ab58bb61 path subv2
v6.1:
# ./btrfs subv list -u /mnt/btrfs/
ID 256 gen 12 top level 5 uuid - path subv1
ID 257 gen 13 top level 5 uuid - path subv2
[CAUSE]
Commit d729048be6 ("btrfs-progs: stop using btrfs_root_item_v0")
removed old btrfs_root_item_v0, but incorrectly changed the check for
v0 root item.
Now we will treat v0 root items as latest root items, causing possible
out-of-bound access, while treating current root items as older v0 root
items, ignoring the UUID nor timestamp.
[FIX]
Fix the bug by using correct checks, and add extra comments on the
branches.
Issue: #562
Fixes: d729048be6 ("btrfs-progs: stop using btrfs_root_item_v0")
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Neal Gompa <neal@gompa.dev>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add initial reflink group with example command 'clone' to test the
interface. Work in progress, experimental build needed.
Issue: #396
Signed-off-by: David Sterba <dsterba@suse.com>
