Commit graph

5606 commits

Author SHA1 Message Date
David Sterba 979bda6fb5 btrfs-progs: libbtrfs: replace SZ_ constants and drop sizes.h
To drop sizes.h from exported headers, replace the few SZ_ constants
from the existing exported headers (ctree.h, send.h). It would be nice
to use them in the long run but right now it would prevent unexporting
the sizes.h file.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:46:35 +02:00
David Sterba 38356d456b btrfs-progs: libbtrfs: drop radix-tree.h from exported headers
The header is only included from ctree.h but not actually used, we can
drop it from the exported files.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:46:35 +02:00
David Sterba aaf13469c0 btrfs-progs: libbtrfs: drop all unneeded build objects
Drop basically all build objects that were entangled due to various
interdependencies and accumulated over the time to libbtrfs.

The commit of shame from 2013 is e5cb128a95 ("btrfs-progs: libify some
parts of btrfs-progs") and let that be a warning how not to do a
library.

The send stream is not yet available in libbtrfsutil, once it will be we
can drop libbtrfs for good.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:46:35 +02:00
David Sterba f404f14917 btrfs-progs: libbtrfs: copy btrfs_uuid_tree_lookup_any from uuid-tree.c
Copy the lookup helper and switch the API functions using it. We can
drop dependency on kernel-shared/uuid-tree.c after that.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:46:35 +02:00
David Sterba 46fa64417c btrfs-progs: libbtrfs: drop btrfs-list.o from build
Nothing from that is needed anymore.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:46:35 +02:00
David Sterba 48be3ad5b3 btrfs-progs: libbtrfs: use btrfs_subvolid_resolve for id->path resolution
There are two functions for resolving subvolume id path, but one is less
convenient and pulls a lot of dependencies. This is
btrfs_list_path_for_root, where the idea is to have the whole tree of
subvolumes and query it as needed.

For simple path resolution we already have btrfs_subvolid_resolve and
it's also in the public API, so we can use it and drop the other one.
This in turn allows to drop btrfs-list.o from build dependencies.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:46:35 +02:00
David Sterba c774b45e22 btrfs-progs: libbtrfs: make own copy of lookup_path_rootid
Remove dependency on utils.h and copy the simple lookup_path_rootid
helper.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:46:35 +02:00
David Sterba d9009ed8ad btrfs-progs: libbtrfs: remove unneeded uuid.h includes
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:46:35 +02:00
David Sterba 22c2fbd86d btrfs-progs: libbtrfs: switch to bare fprintf for error messages
Don't use the internal message helpers and remove the dependency on
messages.h.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:46:34 +02:00
David Sterba 35882be40a btrfs-progs: remove BTRFS_FLAT_INCLUDES from internal send headers
There's a separate copy of the headers, we can remove the include switch
from the internal copies.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:46:34 +02:00
David Sterba 6b5c089469 btrfs-progs: libbtrfs: make own copy of files implementing the API
The two files send-stream and send-utils contain the implementations of
the exported API, which was just for send stream. This was the original
idea. That libbtrfs contains another 40 files was a result of
unclean/missing library design and had to be done that way to resolve
the symbols due to dependencies.

That the same files have been used for both internal and public library
has prevented refactoring and cleanups and was always a risk of breaking
something.

Make separate copy for libbtrfs utils and allow any cleanups and
reduction of number of build objects. The API hasn't changed since the
beginning so there's low risk of missing some fixes from the internal
code.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:46:34 +02:00
David Sterba d4dc4793fa btrfs-progs: build: merge object variables
Now libbtrfs.a has own list and we can merge $objects and
$shared_objects again. Reformat it now that it's changed.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:46:34 +02:00
David Sterba e0f7a9a902 btrfs-progs: build: enumerate libbtrfs.a build objects
Enumerate all the objects required for libbtrfs.a, removing unnecessary
ones on the way

- kernel-lib/radix-tree.o
- kernel-shared/inode-item.o
- libbtrfsutil/stubs.o

There are only a handful of files exported in libbtrfs.sym so the file
is excessively long and most of the code is not necessary. This will be
reduced eventually.

static_libbtrfs_objects need to use the linked shared_objects to avoid
duplication of symbols.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:46:34 +02:00
David Sterba 9a136b39df btrfs-progs: build: replace libbtrfs.a by object list
Avoid using libbtrfs.a as build object so replace it with the same list
of objects.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:46:34 +02:00
David Sterba 1eb3470681 btrfs-progs: build: simplify library variables
As preparation to make libbtrfs build standalone and separate, start
with variables simplifications and inlining.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:46:34 +02:00
Nikolay Borisov 39c6e0b79c btrfs-progs: add btrfs_uuid_tree_remove
It will be used to clear received data on RW snapshots that were
received. The function is copied from kernel sources.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:46:34 +02:00
Nikolay Borisov 97640a5b81 btrfs-progs: remove root argument from btrfs_truncate_item
This function lies in the kernel-shared directory and is supposed to be
close to 1:1 copy with its kernel counterpart, yet it takes one extra
argument - root. But this is now unused to simply remove it.

Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:46:34 +02:00
Nikolay Borisov c3584b4fc0 btrfs-progs: remove fs_info argument from leaf_data_end
The function already takes an extent_buffer which has a reference to
the owning filesystem's fs_info. This also brings the function in line
with the kernel's signature.

Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:46:34 +02:00
Nikolay Borisov 7c58b09548 btrfs-progs: remove root argument from btrfs_fixup_low_keys
It's not used, so just remove it.

Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:46:34 +02:00
David Sterba dad03fac3b btrfs-progs: switch btrfs_group_profile_str to use raid table
Use the profile names from the table, preserving the whole semantics of
unknown flags detection.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:46:34 +02:00
David Sterba 5664631b5b btrfs-progs: clean up test_uuid_unique
Move the declaration to the right header, constify argument and document
the function.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:46:33 +02:00
David Sterba 35f17bc49c
Btrfs progs v5.14.2
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 15:30:04 +02:00
David Sterba 88b2f0c5c1 btrfs-progs: update CHANGES for 5.14.2
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-07 22:00:09 +02:00
David Sterba cb9bd8e326 btrfs-progs: subvol show: print a warning with rw and receive_uuid
Add a slightly more convenient way to identify the subvolumes with bad
combination of flags and received uuid.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-07 18:40:34 +02:00
David Sterba 0a4f9b93ab btrfs-progs: docs: subvolume ro->rw and incremental send
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-07 18:40:30 +02:00
David Sterba 287d472b63 btrfs-progs: tests: subvolume ro->rw switch and received_uuid
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-07 18:40:29 +02:00
David Sterba 3b90ebc2d7 btrfs-progs: prop set: ro->rw and received_uuid
Implement safety check when a read-only subvolume is getting switched
to read-write and there's received_uuid set.

This prevents accidental breakage of incremental send use case but
allows user to do the rw change anyway but resets the received_uuid in
that case.

As this is implemented entirely in userspace, it's racy and using the
raw ioctl won't prevent it nor reset the received_uuid. A change in the
ioctl implementation might do that in the future.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-07 18:40:26 +02:00
David Sterba 332c193c47 btrfs-progs: prop set: add force parameter
Add option support to force the value change. This allows to do safety
checks by default and warn user that something might break. Using the
force will override that and changing the property should do change
itself and additionally any other changes that could break some
use cases.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-07 18:40:24 +02:00
David Sterba 49bd94cc14 btrfs-progs: tests: make misc/038 work with free-space-tree
The test misc-tests/038-backup-root-corruption expects a particular
layout of the backup roots but this gets slightly changed due to free
space tree.

  $ make TEST=038-backup-root-corruption TEST_ARGS_MKFS='-Rfree-space-tree' TEST_ENABLE_OVERRIDE=true test-misc

Will result in test failure as the expected root is in slot 3 and not 2.
Update the test to try both.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-07 18:40:21 +02:00
David Sterba d0ea2b2af4 btrfs-progs: zoned: also exclude raid1c3 and raid1c4 from supported profiles
The enumeration of profiles not available for zoned mode in
btrfs_load_block_group_zone_info was lacking the 3 and 4 copy raid1, add
them.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-07 18:39:58 +02:00
David Sterba 27207d651a btrfs-progs: dump-tree: print complete root_item
The output of root_item in the 'inspect dump-tree' command lacks some
items and some of them are printed conditionally. As the dump utility
is for debugging, it's better to print all the items, with names
matching the structure members and order.

Some values will inevitably be all zeros like uuids or various
timestamps, but that's a minor issue and affecting only a few trees.

Example:

  item 0 key (EXTENT_TREE ROOT_ITEM 0) itemoff 15844 itemsize 439
	  generation 5 root_dirid 0 bytenr 30523392 byte_limit 0 bytes_used 16384
	  last_snapshot 0 flags 0x0(none) refs 1
	  drop_progress key (0 UNKNOWN.0 0) drop_level 0
	  level 0 generation_v2 5
	  uuid 00000000-0000-0000-0000-000000000000
	  parent_uuid 00000000-0000-0000-0000-000000000000
	  received_uuid 00000000-0000-0000-0000-000000000000
	  ctransid 0 otransid 0 stransid 0 rtransid 0
	  ctime 0.0 (1970-01-01 01:00:00)
	  otime 0.0 (1970-01-01 01:00:00)
	  stime 0.0 (1970-01-01 01:00:00)
	  rtime 0.0 (1970-01-01 01:00:00)

  item 3 key (FS_TREE ROOT_ITEM 0) itemoff 14949 itemsize 439
	  generation 4 root_dirid 256 bytenr 30408704 byte_limit 0 bytes_used 16384
	  last_snapshot 0 flags 0x0(none) refs 1
	  drop_progress key (0 UNKNOWN.0 0) drop_level 0
	  level 0 generation_v2 4
	  uuid ec4669b6-6d21-46ab-857e-d60cafde45b3
	  parent_uuid 00000000-0000-0000-0000-000000000000
	  received_uuid 00000000-0000-0000-0000-000000000000
	  ctransid 0 otransid 0 stransid 0 rtransid 0
	  ctime 1633021823.0 (2021-09-30 19:10:23)
	  otime 1633021823.0 (2021-09-30 19:10:23)
	  stime 0.0 (1970-01-01 01:00:00)
	  rtime 0.0 (1970-01-01 01:00:00)

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-07 18:39:44 +02:00
David Sterba ad4d5b163b btrfs-progs: subvol show: print send and receive generation and timestamp
There are some send/receive related data not printed in subvol show,
while they're exported by the ioctls. Print them for convenience:

  $ btrfs subvol show test
  test
	  Name: 		test
	  UUID: 		dc16dd1b-825f-3245-94a8-557672d6cf85
	  Parent UUID: 		-
	  Received UUID: 	-
	  Creation time: 	2021-05-17 16:17:14 +0200
	  Subvolume ID: 	19112
	  Generation: 		7730702
	  Gen at creation:	7730701
	  Parent ID: 		5
	  Top level ID: 	5
	  Flags: 		-
	  Send transid: 	0
	  Send time: 		2021-05-17 16:17:14 +0200
	  Receive transid: 	0
	  Receive time: 	-
	  Snapshot(s):
				  test-snap

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-07 18:39:38 +02:00
David Sterba 42e7b7e617 btrfs-progs: gitignore: update paths in Documentation
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-06 16:50:57 +02:00
David Sterba e1377c073e btrfs-progs: ci: add missing docker scripts for musl image
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-06 16:50:56 +02:00
David Sterba 463ef8515c btrfs-progs: ci: add openSUSE Leap 15.3 image
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-06 16:50:53 +02:00
David Sterba 35d9b3266d btrfs-progs: ci: update base images with libudev-devel
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-06 16:50:51 +02:00
David Sterba 27615e18bc btrfs-progs: ci: disable libudev for musl build
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-06 16:50:50 +02:00
Anand Jain edd57bf494 btrfs-progs: fix comments in cmd_filesystem_show
I had to go back to find what BTRFS_ARG_REG is, add a comment for that.

And, search_umounted_fs_uuids() is also to find the seed device, so bring
the related comment above it.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-06 16:50:46 +02:00
David Sterba 830422fa7e btrfs-progs: build: capitalize variable names in the summary
The CFLAGS and LDFLAGS should be referred to as they appear in the
makefile.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-06 16:50:44 +02:00
David Sterba 1d0fa3781e btrfs-progs: build: make libudev selectable
Previously the build would be optional based on what is installed on the
system. Add an option to make this selectable by user, ie. to allow
build without libudev even if it is installed on the system.

For most users and distros libudev dependency should be ok, but we've
had requests to disable various features (embedded, systems with
non-systemd user space base system) so let's do the same.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-06 16:50:43 +02:00
David Sterba c6c2e3da0a btrfs-progs: device scan: rename is_path_device
The term 'path' is confusing as we normally use it for filesystem paths,
while for multipath it's more related to the physical path by which the
devices are connected (though it also shows up as another path in the
filesystem).

Rename the helper doing the multipath detection so it's clear what path
is meant by that.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-06 16:50:42 +02:00
Nikolay Borisov 1b7128c152 btrfs-progs: add fallback code for multipath device detection for static build
Since libudev doesn't provide a static version of the library for static
build btrfs-progs will have to provide manual fallback. This change does
this by parsing the udev database files hosted at /run/udev/data/.
Under that directory every block device should have a file with the
following name: bMAJ:MIN. So implement the bare minimum code necessary
to parse this file and search for the presence of DM_MULTIPATH_DEVICE_PATH
udev attribute. This could likely be racy since access to the udev
database is done outside of libudev but that's the best that can be
done when implementing this manually and is only for a limited usecase
where static build has to be used.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-06 16:50:37 +02:00
Nikolay Borisov 681b4bd5e1 btrfs-progs: ignore devices representing paths in multipath
Currently btrfs-progs will happily enumerate any device which has a
btrfs filesystem on it irrespective of its type. For the majority of
use cases that's fine and there haven't been any problems with that.
However, there was a recent report that in multipath scenario when
running "btrfs fi show" after a path flap (path going down and then
coming back up) instead of the multipath device being show the device
which represents the flapped path is shown. So a multipath filesystem
might look like:

  Label: none  uuid: d3c1261f-18be-4015-9fef-6b35759dfdba
	  Total devices 1 FS bytes used 192.00KiB
	  devid    1 size 10.00GiB used 536.00MiB path /dev/mapper/3600140501cc1f49e5364f0093869c763

/dev/mapper/xxx is actually backed by an arbitrary number of paths,
which in turn are presented to the system as ordinary SCSI devices i.e
/dev/sdX. If a path flaps and a user re-runs 'btrfs fi show' the output
would look like:

  Label: none  uuid: d3c1261f-18be-4015-9fef-6b35759dfdba
	  Total devices 1 FS bytes used 192.00KiB
	  devid    1 size 10.00GiB used 536.00MiB path /dev/sdd

This only occurs on unmounted filesystems as those are enumerated by
btrfs-progs, for mounted filesystem the kernel properly deals only with
the actual multipath device.

Turns out the output of this command is consumed by libraries and the
presence of a path device rather than the actual multipath causes
issues.

Fix this by checking for the presence of DM_MULTIPATH_DEVICE_PATH
udev attribute as multipath path devices are tagged with this attribute
by the multipath udev scripts.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-06 16:50:35 +02:00
Nikolay Borisov f74ebeee5c btrfs-progs: build: add optional dependency on libudev
This is needed for future code which will make btrfs-progs' device
scanning logic a little smarter by filtering out path device in
multipath setups. libudev is added as an optional dependency since the
library doesn't have a static version so making it a hard dependency
means forfeiting static build support. To alleviate this a fallback code
will be added for the static build case which doesn't rely on libudev.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-06 16:50:32 +02:00
David Sterba 91c91c1632 btrfs-progs: sb-mod: improve help
Print the list of supported members to change and explain what the tool
does.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-06 16:50:30 +02:00
David Sterba f7e4860e9f btrfs-progs: sb-mod: don't fixup checksum for read-only ops
On images with some damage and mismatching checksum the read operations
will fixup the checksum. This may not be desired and does not make much
sense for read anyway.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-06 16:50:28 +02:00
Josef Bacik dd8e7477f7 btrfs-progs: remove data extents from the free space tree
Dave reported a failure of mkfs-test 009 with the free space tree
enabled by default.  This is because 009 pre-populates the file system
with a given directory, and for some reason our data allocation path
isn't the same as in the kernel.  Fix this by making sure when we
allocate a data extent we remove the space from the free space tree, and
with this our mkfs tests now pass.

Issue: #410
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-06 16:49:52 +02:00
David Sterba 64ed09c318 btrfs-progs: mkfs: switch to global verbosity options
Use the bconf verbosity so that other code outside of main.c respects
the command line settings.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-06 16:49:42 +02:00
David Sterba 2814ec6b1f btrfs-progs: mkfs: add option -v/--verbose
The default output of mkfs is intentionally verbose so we did not need
the verbosity option. For some additional information it could be useful
to increase the level in case it's wired to the global verbosity
settings.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-06 16:49:39 +02:00
David Sterba 2d50b98189 btrfs-progs: replace start: add option -K/--nodiscard
The commands initializing a new device (mkfs, device add) do discard by
default, while this is missing from replace start. For parity add the
options with same name and semantics.

Issue: #390
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-06 16:49:35 +02:00