Commit graph

6933 commits

Author SHA1 Message Date
David Sterba eaa54973b6 btrfs-progs: scrub limit: add option to apply the limit to all devices
Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-09 01:57:29 +01:00
David Sterba 30d1a2d390 btrfs-progs: scrub limit: allow to set the limit
Add new options to set the per-device limit (requires root privileges as
it writes to the sysfs files).

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-09 01:57:27 +01:00
David Sterba 87dba20daf btrfs-progs: change all sysfs helpers to return errno
To be consistent with the rest of the code the sysfs helper should
return the -errno instead of passing -1 from various syscalls. Update
callers that relied on -1 as the invalid file descriptor.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-09 01:17:22 +01:00
David Sterba 4576029dfd btrfs-progs: add write helpers for sysfs files
Add convenience wrappers for writing a buffer or u64 to toplevel or FSID
file in sysfs.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-09 01:17:22 +01:00
David Disseldorp b3b38751e9 btrfs-progs: scrub status: improve Rate reporting for sub-second durations
Scrubs which complete in under one second may carry a duration rounded
down to zero. This subsequently results in a bytes_per_sec value of
zero, which corresponds to the Rate metric output, causing intermittent
tests/btrfs/282 failures.

This change ensures that Rate reflects any sub-second bytes processed.
Time left and ETA metrics are also affected by this change, in that they
increase to account for (sub-second) bytes_per_sec.

Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Disseldorp <ddiss@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-09 01:17:22 +01:00
David Sterba 7f869161b7 btrfs-progs: docs: update scrub io limiting
[ci skip]

Issue: #402
Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-09 01:17:22 +01:00
David Sterba 9a044fa0ff btrfs-progs: scrub start: print device limit if set
Print one message per scrubbed device and also print the limit if set:

  $ btrfs scrub start /mnt
  scrub started on /mnt, fsid 9ee93131-f680-4d6c-8ca4-a194506e3081 (pid=27257)
  Starting scrub on devid 1 (limit 100.00MiB/s)

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-09 01:17:22 +01:00
David Sterba 6e96df640b btrfs-progs: tests: add case for scrub limit
Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-09 01:17:22 +01:00
David Sterba fbf211459a btrfs-progs: srcub: new subcommand limit
Add new command to read the scrub limits set via the sysfs file (no root
access needed).

Example output:

  $ btrfs scrub limit /mnt
  UUID: 57a05502-9e81-4b21-ad9d-0fc31863ed11
  Id  Limit            Path
  --  -----  --------------
   1      -  /dev/nvme0n1p1
   2      -  /dev/nvme0n1p2
   3      -  /dev/nvme0n1p3
   4      -  /dev/nvme2n1p4
   5      -  /dev/nvme0n1p5
   6      -  /dev/nvme0n1p6

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-09 01:17:22 +01:00
David Sterba deffef9cec btrfs-progs: README: add compatibility
[ci skip]

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-08 17:08:55 +01:00
David Sterba b5909e62b2 btrfs-progs: use statvfs() in print_filesystem_usage_overall
The statfs(2) syscall is deprecated by LSB in favor of statvfs(2),
however we can't replace all uses because we still need the
statfs::f_type to determine the filesystem by magic numer.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-08 17:08:55 +01:00
David Sterba 71ad6f2f53 btrfs-progs: subvol sync: check if the filesystem is writable
The subvolume cleaning is done by polling but it's possible that the
filesystem turns to read-only (as reported), either due to an error
intentionally. In that case the waiting would be indefinite without an
obvious reason.

To fix that check if the filesystem is still writable in each iteration.

Issue: #535
Link: https://github.com/btrfs/fstests/issues/40
Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-08 17:08:55 +01:00
David Sterba 7e4a235df1 btrfs-progs: scrub status: print device speed limit in status if set
When there's a speed limit set for a device via
/sysfs/fs/btrfs/FSID/devinfo/scrub_speed_max, show it in the scrub status
output like below:

  $ btrfs scrub status -d /mnt
  ...
  Rate:              47.98MiB/s (limit 60MiB/s)
  ...

If the limit is 0 this means unlimited and is not printed.

For a single device filesystem the limit is printed even without '-d' as
it's clear which device limit applies. For multi-device filesysetms,
without any limits nothing is printed, if there at least one device
limit set then the following is printed:

  Rate:             36.37MiB/s (some device limits set)

More details with the -d option.

Issue: #531
Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-08 17:08:33 +01:00
David Sterba f8711016eb btrfs-progs: receive: properly report lack of zstd support
If zstd is not compiled in then a stream fails with a generic error
message:

ERROR: unknown compression: 2

Where BTRFS_ENCODED_IO_COMPRESSION_ZSTD is 2 and there's a case for that
but behind the '#if COMPRESSION_ZSTD'.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-07 14:29:54 +01:00
David Sterba 9aafb384cb btrfs-progs: docs: cross references, ioctl updates
[ci skip]

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-06 17:44:28 +01:00
David Sterba 478a295b00 btrfs-progs: docs: document label ioctls
[ci skip]

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-06 17:44:28 +01:00
David Sterba f9ebee6838 btrfs-progs: ci: install RTD sphinx theme for devel workflow
With the recent updates to documentation build the theme must be now
installed as a package. Disable building documentation in all workflows
that do functional tests.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-06 16:03:37 +01:00
David Sterba c1ce48ed3f btrfs-progs: docs: document device add and remove ioctls
[ci skip]

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-05 20:17:31 +01:00
David Sterba 216c9f0ffb btrfs-progs: docs: move doc conventions to developer docs
The DocConventions are now fairly complete and can be moved to the
proper section.

[ci skip]

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-05 19:33:28 +01:00
Qu Wenruo eae4109054 btrfs-progs: check: remove inode cache clearing functionality
Since we're already directing the end user to use "btrfs rescue
clear-ino-cache" command, there is not much need to support it in
btrfs-check.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-05 17:57:04 +01:00
Qu Wenruo 171dd56995 btrfs-progs: docs: update the man page for btrfs check lowmem mode
Lowmem mode has improved quite a lot since its introduction, for
read-only check it's definitely fine.

For repair mode, both lowmem and original mode are considered dangerous
especially for complex corruptions with unknown cause.

For now lowmem mode is only bad at fixing fundamentally corrupted cases,
like bad shift offsets or transid, which in real world it's not an easy
repair for the original mode either.

This patch would move the --mode option out of the dangerous section and
update the notes for the lowmem mode on its limitation.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-05 17:37:25 +01:00
David Sterba 5c91264d2d btrfs-progs: subvol delete: print the id of the deleted subvolume
Currently the path of deleted subvolume is printed, we should also print
the numeric id as it's another identifier commonly found and can be used
for a cross reference. In connection with the qgroup deletion it's
making the output clear:

...
Delete subvolume 258 (no-commit): '/mnt/subv1'
Delete qgroup 0/258
...

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-05 17:08:42 +01:00
David Sterba 8b9684f965 btrfs-progs: docs: add config file for readthedocs.io
There's another config required for building the RTD documentation,
https://docs.readthedocs.io/en/stable/config-file/ .

[ci skip]

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-05 16:36:24 +01:00
David Sterba cc2e3ab03d btrfs-progs: docs: update and restyle links of source repositories
[ci skip]

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-05 16:19:09 +01:00
David Sterba 71042e8bb0 btrfs-progs: tests: add test for subvolume delete and qgroups
Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-05 12:36:44 +01:00
David Sterba 9da773aa46 btrfs-progs: subvol delete: add options to delete the qgroup
The 0/subvolid qgroups are not automatically deleted when the subvolume
is deleted, for historical reasons. There's a command to clean up all
such stale qgroups (btrfs qgroup clean-stale) but this should be also
possible with the subvolume deletion.

With the options we can switch the default to delete the qgroup by
default eventually, if somebody depends on the not deleting behaviour
the negation option can be used.

Issue: #366
Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-05 12:36:44 +01:00
David Sterba a396f49cb8 btrfs-progs: build: conditionally detect x86_64 compiler flags
Compiling with clang on aarch64 leads to an error when detecting the
SIMD instruction support. Gcc ignores the arch/feature mismatch.
Conditionally detect the -m flags only on x86_64.

Issue: #712
Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-05 12:36:44 +01:00
David Sterba bc70e12877 btrfs-progs: ci: add clang to devel build tests
Add clang as compiler for the basic build checks in the CI.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-04 18:55:47 +01:00
David Sterba 4ac630b5dc btrfs-progs: ci: update packages providing pahole
Centos 8 does not provide the package at all, on musl it's in the
package 'pahole'.

Issue: #710
Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-04 18:47:31 +01:00
David Sterba 8ea9a3e2c6 btrfs-progs: print a message when enqueued operation is waiting
The enqueue option should let the user know that the expected operation
hasn't started yet and that it's waiting for another one. Although the
exclusive operations can take long, the two reason should be
distinguished.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-01 03:22:29 +01:00
David Sterba c0592c5591 btrfs-progs: tests: warning and timeout when removing multiple devices
Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-01 03:04:58 +01:00
David Sterba 9e55abfd77 btrfs-progs: device delete: add timeout when removing multiple devices
Reported on IRC, that it's unexpected that passing several devices on
command line for 'btrfs device delete' still uses some of the devices
during deletion. The expectation was that they'd be removed at once (and
thus not used for the intermediate chunk relocation).

As it works now, the ioctl removes only one device. As a workaround, add
a timeout (like we have for the full balance and others) when there are
more devices passed on the command line. This can be skipped by the
--force parameter.

Issue: #708
Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-01 02:54:09 +01:00
David Sterba a0468efe86 btrfs-progs: reset errno before strtoull()
strtoull may return the boundary values, if the callers could expect
that and verify it then the errno must be reset before the call.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-01 02:13:37 +01:00
David Sterba e9a66cde91 btrfs-progs: use get_sysfs_proto_supported() in device_get_zone_unusable()
Use the sysfs helper to read u64 values of the zoned stats.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-01 02:08:37 +01:00
David Sterba 9da7e92af9 btrfs-progs: use get_sysfs_proto_supported() in get_sysfs_proto_supported()
Use the sysfs helper to read u64 value of send stream.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-01 01:58:56 +01:00
David Sterba 22495fc8fd btrfs-progs: tests: add coverage for size options of 'fi du'
Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-01 01:46:05 +01:00
David Sterba b512558c02 btrfs-progs: build: add target for all internal APIs
There are several APIs that have tests, add a single build target for
convenience and enable that in the CI.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-01 01:39:27 +01:00
David Sterba 960e7b3800 btrfs-progs: tests: test log levels
The log levels have been added in 6.2, add the basic coverage.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-01 01:32:40 +01:00
David Sterba 41781638f9 btrfs-progs: README: document static builds
[ci skip]

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-01 01:22:06 +01:00
David Sterba 0c155146c3 btrfs-progs: README: update links
[ci skip]

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-01 01:08:08 +01:00
David Sterba ac8edc1513 btrfs-progs: docs: restyle the landing page
Add an introductory paragraph with most important links, put TOC
listings to columns.

[ci skip]

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-01 00:54:27 +01:00
David Sterba 32880fa518 btrfs-progs: crypto: add openssl as crypto provider
https://www.openssl.org/ Is a well known cryptography library and since
freshly released version 3.2 it also supports variable digest size of
blake2b, so we can now add it among the crypto providers.

Configure with --with-crypto=openssl.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-30 22:04:45 +01:00
David Sterba 5221aedc00 btrfs-progs: crypto: add Botan as crypto provider
https://botan.randombit.net/ Botan is a cryptography library with C
bindings and provides what we need (sha256 and blake2b), among many
others. Add it to the list of crypto backends if somebody wants to use
it.

Currently the version 2.19 is the latest one. Botan3 3.2.0 exists but
does not seem to be widely available in distros yet.

Configure with --with-crypto=botan.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-30 22:04:45 +01:00
Qu Wenruo d79eb4b726 btrfs-progs: tests: add test case for subvolume create multiple arguments
The test case would verify the following behaviors:

- Partial failure
  Should return 1, but the remaining valid destinations would still be
  created.

- All success
  That's as usual.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-23 20:21:58 +01:00
Qu Wenruo 5aa959fb34 btrfs-progs: subvolume create: accept multiple arguments
This patch would make "btrfs subvolume create" to accept multiple
arguments, just like "mkdir".

The existing options like "-i <qgroupid>" and "-p" would all be applied
to all subvolume(s).

If one destination failed, the command would return 1, while still retry
the remaining destinations.

Issue: #695
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-23 20:15:47 +01:00
Qu Wenruo 943a69ef2d btrfs-progs: subvolume create: handle failure for strdup()
The function strdup() can return NULL if the system is out of memory,
thus we need to hanle the rare but possible -ENOMEM case.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-23 20:11:17 +01:00
David Sterba 69a3871fdd btrfs-progs: build: fix reported version of crypto libraries
The option --version actually reports version of pkg-config not the
module/library we're interested in.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-23 16:20:28 +01:00
David Sterba afa23d4e27
Btrfs progs v6.6.2
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-15 15:56:12 +01:00
David Sterba c2b77c03a5 btrfs-progs: update CHANGES for 6.6.2
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-15 15:55:01 +01:00
David Sterba 4d7ea19ca0 btrfs-progs: property set: skip opening char devices completely
Previous fix for char devices and properties opens the path in non
blocking mode but this still triggers the watchdog, as reported. Add a
workaround to properties to completely skip opening the path and just
stat() it.

Issue: #699
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-14 15:48:00 +01:00