Commit graph

6735 commits

Author SHA1 Message Date
Qu Wenruo 93e8b661f6 btrfs-progs: fix a variable shadowing when enabling experimental features
There is another variable shadowing problem which can only be exposed
if experimental features are enabled.

Inside the branch of BTRFS_PRINT_TREE_CSUM_HEADERS, we declare another
local variable @csum, shadowing the @csum of print_header_info(), which
is only declared when experimental features are enabled.

Just rename the @csum to @tree_csum to avoid the problem.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-02 18:41:08 +02:00
Qu Wenruo c94965198a btrfs-progs: replace errno with %m in cmd_inspect_list_chunks()
The variable @e is only utilized to record the errno from ioctl() call,
and is only for the error message.

We can go with "%m" to replace the usage of variable @e, and remove the
variable shadowing, as later we will declare a local variable @e with a
different type.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-02 18:41:08 +02:00
David Sterba 553273a834 btrfs-progs: dump-tree: fix block group tree short name
The option -t recognizes tree names in various names and the stem must
not contain the "_TREE", which the bgt had.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-02 18:41:08 +02:00
Johannes Thumshirn a9fd50f85f btrfs-progs: read stripe tree when mapping blocks
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-02 18:41:08 +02:00
Johannes Thumshirn fff57d3774 btrfs-progs: load zone info for all zoned devices
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-02 18:41:08 +02:00
Johannes Thumshirn b4ab282686 btrfs-progs: allow zoned RAID
Allow for RAID levels 0, 1 and 10 on zoned devices if the RAID stripe tree
is used.

Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-02 18:41:08 +02:00
Johannes Thumshirn 81d6ecd5c2 btrfs-progs: add dump tree support for the raid stripe tree
Add support for the RAID stripe tree to btrfs inspect-internal dump-tree.

Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-02 18:41:08 +02:00
Johannes Thumshirn b6490733a8 btrfs-progs: read fs with stripe tree from disk
When encountering a filesystem formatted with the raid stripe tree
feature, read it from disk.

Also add the incompat declaration to the tree printer.

Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-02 18:41:08 +02:00
Johannes Thumshirn 2eb351210f btrfs-progs: add raid stripe tree definitions
Add the definitions for the on-disk format of the raid stripe tree.

Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-02 18:41:08 +02:00
David Sterba 8d3193da89 btrfs-progs: increase size of BTRFS_FEATURE_STRING_BUF_SIZE
We're getting more features and the string size limit will not be
sufficient, so extend enough that we won't have to care for some time.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-02 18:41:08 +02:00
David Sterba 7db445a3de btrfs-progs: crypto: unify naming of crc32c implementations
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-02 18:05:38 +02:00
David Sterba 78c227e7d2 btrfs-progs: ci: run CI image tests in parallel
We can speed up the CI tests by running them as separate build jobs,
they only need to checkout the sources.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-02 18:05:38 +02:00
David Sterba 92d04d4780
Btrfs progs v6.5.1
Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-13 19:47:53 +02:00
David Sterba 2c574064b5 btrfs-progs: update CHANGES for 6.5.1
Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-13 19:47:10 +02:00
David Sterba d0bf814100 btrfs-progs: ci: build static binaries only for latest release
The release-test branch is used for pre-release checks so don't pollute
the workflows list with unverified builds. The static build checks will
be done within the other workflows but no binaries need to be published.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-13 19:43:26 +02:00
David Sterba 03f41ac508 btrfs-progs: detect PCLMUL CPU support for accelerated crc32c
The accelerated crc32c needs to check for two CPU features, the crc32c
instructions is in SSE 4.2 and 'pclmulqdq' is a separate. There's still
old hardware used that does not have the PCLMUL instructions. Detect it
and make it the condition.

The pclmul is not supported on old compilers so also add a
configure-time detection and leave the SSE 4.2 only implementation as
the accelerated one if possible.

Issue: #676
Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-13 00:38:50 +02:00
David Sterba b40943dea4 btrfs-progs: docs: updates
- group features on status page
- update developer docs
- add cross references

Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-12 23:18:35 +02:00
David Sterba 2726a83952 btrfs-progs: tests: scan test results
Some errors may be reported in the logs only, scan the file each time
there are fresh results. The scanning script will return error that is
supposed to be caught by the testsuite environment.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-12 23:18:35 +02:00
David Sterba 7a7df06f98 btrfs-progs: tests: extend scan-results.sh to scan only a given file
Scanning all results is meant for the whole testsuite, we'd like to make
it more fine grained to verify them once a test is run via the test
running wrappers.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-12 23:18:35 +02:00
David Sterba e0263f2b31 btrfs-progs: tests: fix test case name detection scan-results.sh
The test markers have changed in 4.17 so the result scanner does not
print the test names (but still detects the errors).

Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-12 23:18:34 +02:00
David Sterba 634e9a4792 btrfs-progs: tests: fix build -Wmissing-prototypes warnings
Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-12 23:18:34 +02:00
Mark Harmstone d8317a445c btrfs-progs: check: add check for metadata level mismatch
For a skinny metadata item in the extent tree, the key offset represents
the level of the tree it points to. This adds a check that these values
match, as otherwise it can cause a volume to go readonly when deleting a
large number of inodes.

See https://github.com/maharmstone/ntfs2btrfs/issues/51

Pull-request: #623
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Mark Harmstone <mark@harmstone.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-12 23:18:34 +02:00
Ariadne Conill b390c276c2 btrfs-progs: crypto: fix readonly relocation of the jumptable
Without this, the btrfs programs fail to link when build with -Wl,-z,relro,
due to the jumptable containing relocations.

Pull-request: #675
Signed-off-by: Ariadne Conill <ariadne@dereferenced.org>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-12 22:08:02 +02:00
David Sterba f7ecc34555
Btrfs progs v6.5
Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-01 14:41:37 +02:00
David Sterba 226f3a97e7 btrfs-progs: update CHANGES for 6.5
Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-01 13:58:46 +02:00
David Sterba f9b0da8e78 Revert "libbtrfs: remove the support for fs without uuid tree"
This reverts commit 83ab92512e.

The commit removed some old code that seemed to be unused but this
actually broke snapper. Revert the changes completely to the v6.3 ABI
level.

Link: https://bugzilla.opensuse.org/show_bug.cgi?id=1212217
Issue: #672
Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-01 13:58:46 +02:00
Qu Wenruo 7a0da24f52 btrfs-progs: tune: do not allow multiple incompatible features to be set
[PROBLEM]
Btrfstune allows multiple different options to be executed in one go,
some options are completely fine, like no-holes along with extref, but
with more and more options, we need more exclusive checks.

In fact a lot of new options are already not following the old
success/total checks.

[ENHANCEMENT]
There is really no need to allow multiple features to be set in one go.

So this patch introduces an array which groups all the compatible
options into following categories:

- Extent tree
  This includes converting to/from extent and block group tree.

- Space cache
  This includes converting to v2 space cache.

- Metadata UUID
  This includes changing metadata uuid.

- FSID change
  This includes the slower full fs fsid rewrites.

- Csum change
  This includes the csum rewrites.

- Seed devices
  This includes changing the device seed flag.

- Legacy options
  This includes no-holes/extref/skinny-metadata features, which are
  already default mkfs features.

Now we only allow options inside the same group to be specified.
E.g. "btrfstune -r -S 1" would fail as it includes both legacy and seed
groups.
Meanwhile "btrfstune -r -n" would still be allowed.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-01 13:58:46 +02:00
Anand Jain e8e5537d8c btrfs-progs: tests: add prefixes to error messages in misc-test/034
Add appropriate prefix to the error messages to make it easier to track
down which case failed.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-01 13:58:46 +02:00
David Sterba bdfb83e7d3 btrfs-progs: README: pull request workflow, minor fixes
Mention the basic pull request workflow, the rest are style fixes.
[skip ci]

Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-01 13:58:46 +02:00
David Sterba 008ca33f3e btrfs-progs: docs: add 6.5 kernel development statistics
Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-01 13:58:41 +02:00
David Sterba 4c24d928d4 btrfs-progs: ci: build static binaries on oldest supported x86-64 architecture
As mentioned on https://wiki.tnonline.net/w/Btrfs/Statically_built_btrfs-progs
the static binaries could be built on with flags so they're available on
most x86-64 models (intel, AMD).

Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-01 12:11:59 +02:00
David Sterba 4167c86123 btrfs-progs: remove btrfs-fragments
The btrfs-fragments utility generates a html page with a set of images
representing block groups and visualises fragmentation. Though this
might be useful, the build has been broken since commit 877f512c55
("btrfs-progs: sync block group item accessors from kernel"), version
5.7. Also it's been out of date regarding newly added trees or data
structure updates and does not work correctly anymore.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-01 12:11:59 +02:00
David Sterba b2c4a9ea75 btrfs-progs: ci: add awk to Tumbleweed image
The CI image build workflow started to fail due to missing 'awk',
install it.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-01 12:11:46 +02:00
David Sterba f58694e87a btrfs-progs: docs: update Interoperability
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:58 +02:00
David Sterba 65306da75c btrfs-progs: ci: add workflow for pull-request
Reduced version of devel checks, unrestricted.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:58 +02:00
Anand Jain 09351e3831 btrfs-progs: rename fs_devices::latest_trans to match the kernel
Aligning progs's struct btrfs_fs_devices with the kernel rename
btrfs_fs_devices::latest_trans to btrfs_fs_devices::latest_generation.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:58 +02:00
Anand Jain a11d468e98 btrfs-progs: rename fs_devices::list to match the kernel
Aligning with the kernel's struct btrfs_fs_devices:fs_list, rename
btrfs_fs_devices::list to btrfs_fs_devices::fs_list.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:58 +02:00
Anand Jain 66c4c9632f btrfs-progs: prepare the latest device's superblock for commit
Add a flag to copy the superblock of the latest device to the
fs_info::super_copy for the commit process, rather than using the
superblock from the device specified in the argument.

This serves as groundwork to enable recovery from an incomplete
btrfstune -M|m|u|U operation.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:58 +02:00
Anand Jain 9f6e0ab2a4 btrfs-progs: tune: fix return without flag reset commit
In the function set_metadata_uuid(), we set the flag
BTRFS_SUPER_FLAG_CHANGING_FSID_V2 in step 1 at line 71 as shown below:

   71         super_flags |= BTRFS_SUPER_FLAG_CHANGING_FSID_V2;
   72         btrfs_set_super_flags(disk_super, super_flags);
   73         ret = btrfs_commit_transaction(trans, root);

However, we fail to reset this flag if there is no change in the fsid on
the incoming disks, as we return too early.

  105       } else {
  106               /* Setting the same fsid as current, do nothing */
  107               return 0;

Fix this by allowing the thread to pass through the step 2, where we
reset the flag.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:58 +02:00
Anand Jain fd3c4c4df1 btrfs-progs: tune: pass metadata_uuid in check_unfinished_fsid_change
In preparation to use check_unfinished_fsid_change() to support the
ability to reunite devices after a failed 'btrfstune -m|M' command,
rename %unused2 to %metadata_uuid as the function
check_unfinished_fsid_change() write the metadata_uuid from the ctree to
it.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:58 +02:00
Anand Jain aff1f16deb btrfs-progs: tune: pass fsid in check_unfinished_fsid_change arg2
In preparation to use check_unfinished_fsid_change() to support the
ability to reunite devices after a failed 'btrfstune -m|M' command,
delete unused1 argument instead reuse %fsid as the function
check_unfinished_fsid_change() returns the fsid.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:58 +02:00
Anand Jain 70bedfa68a btrfs-progs: tune: rename uuid_changed to fsid_changed in set_metadata_uuid
We never change the metadata_uuid; we only change the fsid.  So
'%fsid_changed' flows more appropriately than '%uuid_changed'.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:58 +02:00
Anand Jain c6ecbe54bf btrfs-progs: tune: rename new_uuid to new_fsid in set_metadata_uuid
%new_uuid is being used to say there is a new fsid. So why not just call
it %new_fsid.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:58 +02:00
Anand Jain 21440a64f7 btrfs-progs: tune: rename new_fsid to fsid in set_metadata_uuid
The %new_fsid is not only new it can be the fsid from the passed disk
so just rename it to %fsid. Also, in the next patch the %new_fsid will
be a bool variable to indicate if the %fsid is new from the fsid in the
disk.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:58 +02:00
Anand Jain 9e9ac97902 btrfs-progs: tune: rename arg to new_fsid_str in set_metadata_uuid
In preparation to use check_unfinished_fsid_change() to support the
ability to reunite devices after a failed 'btrfstune -m|M' command,
%uuid_string arg is actually carries new fsid to be used. So just name
it to %new_fsid_str.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:58 +02:00
Anand Jain 3aa3342959 btrfs-progs: tune: cache local variable of fs_info
Since the root pointer dereferences for the fs_info several times,
it is rational to save the fs_info.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:56 +02:00
Anand Jain 4a253f713e btrfs-progs: track num_devices per fs_devices
Similar to the kernel we need to track the number of devices scanned
per fs_devices. A preparation patch to fix incomplete fsid changing.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:25 +02:00
David Sterba 5c7f36b141 btrfs-progs: tests: fix typo in README
The correct helper name is check_global_prereq, this was once
seen accidentally used in a test.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:25 +02:00
Josef Bacik 0ba50a27f4 btrfs-progs: fix improper error handling in btrfs filesystem usage
I was seeing test-cli/016 failures because it claimed we were getting
EPERM from the TREE_SEARCH ioctl to get the chunk info out of the file
system.  This turned out to be because errno was already set going into
this function, the ioctl itself wasn't actually failing.  Fix this by
checking for a return value from the ioctl first, and then returning
-EPERM if appropriate.  This fixed the failures in my setup.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:25 +02:00
Josef Bacik db7157d912 btrfs-progs: clear root dirty when we update the root
We don't currently use the bit to track whether or not the root is
dirty, but when we sync ctree.c it uses this bit to determine if we
should add the root to the dirty list.  Clear this bit when we update
the root so that the dirty tracking works properly when we sync ctree.c.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:25 +02:00