Commit graph

6248 commits

Author SHA1 Message Date
David Sterba 1ab556bed4 btrfs-progs: update CHANGES for 6.1.1
Signed-off-by: David Sterba <dsterba@suse.com>
2023-01-03 17:26:22 +01:00
David Sterba 73efc566d5 btrfs-progs: kerncompat: hide definition of __init
As kerncompat.h is included from all libbtrfs headers we must be careful
about generic names like __init, in this case it breaks build of
snapper.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-01-03 17:24:36 +01:00
David Sterba e78fe2d92e Revert "btrfs-progs: rename qgroup items to match the kernel naming scheme"
This reverts commit 03451430de.
(It's not 1:1, there are some additional trivial fixups in cmds/qgroup.c)

This breaks a lot of 3rd party tools that depend on it as Neal reports:

* btrfs-assistant
* buildah
* cri-o
* podman
* skopeo
* containerd
* moby/docker
* snapper
* source-to-image

Link: https://lore.kernel.org/linux-btrfs/CAEg-Je8L7jieKdoWoZBuBZ6RdXwvwrx04AB0fOZF1fr5Pb-o1g@mail.gmail.com/
Reported-by: Neal Gompa <ngompa@fedoraproject.org>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-01-03 13:10:54 +01:00
Qu Wenruo c339846763 btrfs-progs: misc-tests: add a test case to make sure uuid is correctly reported
The new test case will execute "btrfs subvolume list -u" on the newly
create btrfs.

Since the v0 root item is already deprecated for a long time, newly
created btrfs should be already using the new root item, thus "btrfs
subvolume list -u" should always report the correct uuid.

The test case relies on external program "uuidparse" which should be
provided by util-linux.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-01-02 14:00:10 +01:00
Qu Wenruo b7e3acd615 btrfs-progs: subvol list: fix the wrong timestamp and UUID check for root items
[BUG]
Since commit d729048be6 ("btrfs-progs: stop using
btrfs_root_item_v0"), "btrfs subvolume list -u" not longer correctly
reports UUID nor timestamp, while older (btrfs-progs v6.0.2) still works
correctly:

 v6.0.2:
 # btrfs subv list -u  /mnt/btrfs/
 ID 256 gen 12 top level 5 uuid ed4af580-d512-2644-b392-2a71aaeeb99e path subv1
 ID 257 gen 13 top level 5 uuid a22ccba7-0a0a-a94f-af4b-5116ab58bb61 path subv2

 v6.1:
 # ./btrfs subv list -u /mnt/btrfs/
 ID 256 gen 12 top level 5 uuid -                                    path subv1
 ID 257 gen 13 top level 5 uuid -                                    path subv2

[CAUSE]
Commit d729048be6 ("btrfs-progs: stop using btrfs_root_item_v0")
removed old btrfs_root_item_v0, but incorrectly changed the check for
v0 root item.

Now we will treat v0 root items as latest root items, causing possible
out-of-bound access, while treating current root items as older v0 root
items, ignoring the UUID nor timestamp.

[FIX]
Fix the bug by using correct checks, and add extra comments on the
branches.

Issue: #562
Fixes: d729048be6 ("btrfs-progs: stop using btrfs_root_item_v0")
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Neal Gompa <neal@gompa.dev>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-01-02 13:57:29 +01:00
David Sterba cde5faa678
Btrfs progs v6.1
Signed-off-by: David Sterba <dsterba@suse.com>
2022-12-22 21:01:25 +01:00
David Sterba 9c76e3a4ff btrfs-progs: update CHANGES for 6.1
Signed-off-by: David Sterba <dsterba@suse.com>
2022-12-22 20:49:54 +01:00
David Sterba 3a5adfb1a4 btrfs-progs: docs: add some kernel 6.1 release notes
Signed-off-by: David Sterba <dsterba@suse.com>
2022-12-22 19:01:45 +01:00
David Sterba 4f7bf100a9 btrfs-progs: docs: typo fixups and formatting updates
Signed-off-by: David Sterba <dsterba@suse.com>
2022-12-22 18:44:47 +01:00
Qu Wenruo b80c1d0c7f btrfs-progs: corrupt-block: fix the mismatch in --root and -r options
[BUG]

The following command will crash:

 $ btrfs-corrupt-block --value 4308598784 --root 5 --inode 256 --file-extent 0 \
	-f disk_bytenr ~/test.img

[CAUSE]
The backtrace is at the following code:

			case 'r':
				root_objectid = arg_strtou64(optarg);
				break;

And @optarg is NULL.

The root cause is, for short option "-r" it indeed requires an argument.
But unfortunately for the longer version, it goes:

			{ "root", no_argument, NULL, 'r'},

Thus it gave @optarg as NULL if we go the longer option and crash.

[FIX]
Just fix the argument requirement for "--root" option.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-12-20 20:49:13 +01:00
David Sterba 5684d2f3d9 btrfs-progs: device stats: fix json formatter type for devid
Devid is u64 but the formatter is just for u32.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-12-16 00:49:07 +01:00
David Sterba 62d818e365 btrfs-progs: use escaped json format for paths
Use the new escaped formatter for paths in command 'device stats' and
'qgroup show'.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-12-16 00:43:25 +01:00
David Sterba ecbb6a7fcd btrfs-progs: add json formatter for escaped string
Add formatter type 'str' where the string must be escaped, e.g. paths or
internal data. Otherwise plain %s can be printed if it's known that
there are no special characters.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-12-16 00:40:32 +01:00
David Sterba 6ecaa3ee4f btrfs-progs: qgroup show: add json output
Support json for 'qgroup show' with values printed by "btrfs qgroup
-pcre", the accounted size and the limits.  It's implemented as a
separate call and not sharing the printing routines so any visible
changes need to by synchronized.

Formatter updates: don't print key name if .out_json is NULL.

Example output:

 # btrfs --format json qgroup show /mnt/path
{
  "__header": {
    "version": "1"
  },
  "qgroup-show": [
    {
      "qgroupid": "0/5",
      "referenced": "8831393792",
      "max_referenced": "none",
      "exclusive": "8224075776",
      "max_exclusive": "none",
      "path": "",
      "parents": [
      ],
      "children": [
      ]
    },
    {
      "qgroupid": "0/361",
      "referenced": "611459072",
      "max_referenced": "none",
      "exclusive": "65536",
      "max_exclusive": "none",
      "path": "subv1",
      "parents": [
        "1/1"
      ],
      "children": [
      ]
    },
    {
      "qgroupid": "0/362",
      "referenced": "611459072",
      "max_referenced": "none",
      "exclusive": "65536",
      "max_exclusive": "none",
      "path": "snap1-r",
      "parents": [
      ],
      "children": [
      ]
    },
    {
      "qgroupid": "1/1",
      "referenced": "611459072",
      "max_referenced": "none",
      "exclusive": "65536",
      "max_exclusive": "none",
      "path": "",
      "parents": [
      ],
      "children": [
        "0/361"
      ]
    }
  ]
}

Issue: #555
Signed-off-by: David Sterba <dsterba@suse.com>
2022-12-14 23:42:03 +01:00
David Sterba 8cec98cb75 btrfs-progs: docs: updates, clarifications
Update spelling, add notable kernel/version features or updates.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-12-14 02:56:16 +01:00
David Sterba 719b5a592f btrfs-progs: docs: add 6.1 development statistics
Signed-off-by: David Sterba <dsterba@suse.com>
2022-12-13 16:15:07 +01:00
David Sterba c0360b4735 btrfs-progs: docs: fix typos
Namely change eg. to e.g. and ie. to i.e.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-12-07 21:00:25 +01:00
David Sterba 900d56f1b9 btrfs-progs: add new group reflink and command
Add initial reflink group with example command 'clone' to test the
interface. Work in progress, experimental build needed.

Issue: #396
Signed-off-by: David Sterba <dsterba@suse.com>
2022-12-07 01:43:30 +01:00
David Sterba 427bf75a1e btrfs-progs: fi mkswapfile: update help text
The help text of mkswapfile was a copy&paste leftover, fix it.

Issue: #554
Signed-off-by: David Sterba <dsterba@suse.com>
2022-12-06 23:19:13 +01:00
David Sterba f652a5b754 btrfs-progs: docs: swapfile and hibernation
Document the new swapfile commands and how it could be used for hibernation.

Issue: #533
Issue: #544
Signed-off-by: David Sterba <dsterba@suse.com>
2022-12-06 23:19:12 +01:00
David Sterba 0b3df991db btrfs-progs: fi df: implement json output
Print JSON output with example output:

  $ btrfs --format json filesystem df /path
  {
    "__header": {
      "version": "1"
    },
    "filesystem-df": [
      {
	"bg-type": "Data",
	"bg-profile": "single",
	"total": "841813590016",
	"used": "494085484544"
      },
      {
	"bg-type": "System",
	"bg-profile": "single",
	"total": "33554432",
	"used": "114688"
      },
      {
	"bg-type": "Metadata",
	"bg-profile": "single",
	"total": "24696061952",
	"used": "5743165440"
      },
      {
	"bg-type": "GlobalReserve",
	"bg-profile": "single",
	"total": "536870912",
	"used": "0"
      }
    ]
  }

Optionally there's "zone_unusable" key. The rowspec cannot print the
same output as now so the original implementation is preserved.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-12-06 21:03:31 +01:00
David Sterba b16b35e074 btrfs-progs: inspect: new command map-swapfile
Verify if a given file is suitable for a swapfile and print the physical
offset (ie. the ultimate on-device physical offset), and the resume
offset value (physical / page size).

This can be the kernel parameter or written to /sys/power/resume_offset
before hibernation.  Option -r or --resume-offset prints just the value.

Copied and simplified from Omar Sandoval's tool to print extents:

https://github.com/osandov/osandov-linux/blob/master/scripts/btrfs_map_physical.c

Issue: #544
Issue: #533
Signed-off-by: David Sterba <dsterba@suse.com>
2022-12-06 21:03:31 +01:00
David Sterba 9793474e22 btrfs-progs: tests: add string-table test framework
Signed-off-by: David Sterba <dsterba@suse.com>
2022-12-05 20:32:02 +01:00
David Sterba 8195508f37 btrfs-progs: string-table: add ranged API for printing and clearing
Track how many rows belong to a header (names and separators) so they
can be printed or cleared separately. The rest is body that could be
cleared and filled repeatedly. The ranged API allows to print any number
of rows if the table is filled partially.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-12-05 20:32:02 +01:00
David Sterba b397e0eb63 btrfs-progs: string-table: check bounds before writing to a cell
Signed-off-by: David Sterba <dsterba@suse.com>
2022-12-05 20:32:02 +01:00
David Sterba 9cae43183e btrfs-progs: string-table: cleanup and enhance formatting capabilities
Cleanups are for integer types, prototypes and comments.

New functionality: spacing can be set after table allocation as
->spacing, now able to print 1 or 2 spaces between columns.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-12-05 20:32:02 +01:00
David Sterba 99979a53bb btrfs-progs: inspect: new subcommand to list chunks
New command 'btrfs inspect-internal list-chunks' will list layout of
chunks as stored on the devices. This corresponds to the physical
layout, sorted by the physical offset. The block group usage can be
shown as well, but the search is too slow so it's off by default.

If the physical offset sorting is selected, the empty space between
chunks is also shown.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-12-05 20:28:35 +01:00
Josef Bacik e380421ff2 btrfs-progs: make write_extent_buffer take a const eb
This is what we do in the kernel, and while we're syncing individual
files we're going to have state where some callers are using a const,
but progs isn't.  So adjust write_extent_buffer to take a const eb in
order to make this less painful.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-30 19:14:29 +01:00
Josef Bacik 5324ca5ae0 btrfs-progs: don't use btrfs_header_csum helper
This is a useless helper, the csum is always the first thing in the
header, simply read from offset 0.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-30 19:14:29 +01:00
Josef Bacik 405b5cadd3 btrfs-progs: replace btrfs_leaf_data with btrfs_item_nr_offset
We're using btrfs_item_nr_offset(leaf, 0) to get the start of the leaf
data in the kernel, we don't have btrfs_leaf_data.  Replace all
occurrences of btrfs_leaf_data() with btrfs_item_nr_offset(leaf, 0) in
order to make syncing accessors.[ch] easier.  ctree.c will be synced
later, so this is simply an intermediate step.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-30 19:14:29 +01:00
Josef Bacik 788a71c16a btrfs-progs: sync compression.h from the kernel
This patch copies in compression.h from the kernel.  This is relatively
straightforward, we just have to drop the compression types definition
from ctree.h, and update the image to use BTRFS_NR_COMPRESS_TYPES
instead of BTRFS_COMPRESS_LAST, and add a few things to kerncompat.h to
make everything build smoothly.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-30 19:14:29 +01:00
Josef Bacik fac1fae3ef btrfs-progs: rename extent buffer flags to EXTENT_BUFFER_*
We have been overloading the extent_state flags for use on the extent
buffers as well.  When we sync extent-io-tree.[ch] this will become
impossible, so rename these flags to EXTENT_BUFFER_* and use those
definitions instead of the extent_state definitions.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-28 18:57:44 +01:00
Josef Bacik 83cc5a5489 btrfs-progs: delete state_private code
We used to store random private things into extent_states, but we
haven't done this for a while and there are no users of this code,
simply delete it.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-28 18:57:44 +01:00
Josef Bacik 20d88c17e7 btrfs-progs: move extent cache code directly into btrfs_fs_info
We have some extra features in the btrfs-progs copy of the
extent_io_tree that don't exist in the kernel.  In order to make syncing
easier simply move this functionality into btrfs_fs_info, that way we
can sync in the new extent_io_tree code and not have to worry about
breaking anything.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-28 18:57:44 +01:00
Josef Bacik 412eea9e97 btrfs-progs: do not pass io_tree into verify_parent_transid
We do not use the io_tree, don't bother passing it into
verify_parent_transid.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-28 18:57:44 +01:00
Josef Bacik ccee633f3a btrfs-progs: move dirty eb tracking to it's own io_tree
btrfs-progs has a cache tree embedded in the extent_io_tree in order to
track extent buffers.  We use the extent_io_tree part to track dirty,
and the cache tree to keep the extent buffers in.  When we sync
extent-io-tree.[ch] we'll lose this ability, so separate out the dirty
tracking into its own extent_io_tree.  Subsequent patches will adjust
the extent buffer lookup so it doesn't use the custom extent_io_tree
thing.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-28 18:57:43 +01:00
Josef Bacik af30cf2e3e btrfs-progs: make the find extent buffer helpers take fs_info
This is a cleanup patch to make syncing the btrfs kernel code into
btrfs-progs easier.  In btrfs-progs we have an extra cache in the
extent_io_tree that's exclusively used for the extent buffer tracking.
In order to untangle this dependency start passing around the fs_info to
search for extent_buffers, and then have the helpers use the appropriate
structure to find the extent buffer.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-28 18:57:43 +01:00
Josef Bacik d729048be6 btrfs-progs: stop using btrfs_root_item_v0
This isn't defined in the kernel, we simply check if the root item size
is less than btrfs_root_item, so adjust the user of btrfs_root_item_v0
to make a similar check.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-28 18:57:43 +01:00
Josef Bacik 5fc1d0cd64 btrfs-progs: copy ioctl.h into libbtrfs
We're going to sync btrfs.h into btrfs-progs from the kernel, however
libbtrfs still needs ioctl.h.  To deal with this copy ioctl.h into
libbtrfs, and update that code to use the local copy and update the
libbtrfs headers list to use this copy.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-28 18:57:43 +01:00
Josef Bacik 2b9fba7974 btrfs-progs: rename btrfs_item_end to btrfs_item_data_end
This matches what we did in the kernel, btrfs_item_data_end is more
inline with what the helper does, which is give us the offset of the end
of the data portion of the item, not the offset of the end of the item
itself.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-28 18:57:43 +01:00
Josef Bacik 27db5d147f btrfs-progs: image: rename BLOCK_* to IMAGE_BLOCK_* for metadump
When we sync the kernel we're going to pull in the fs.h dependency,
which defines BLOCK_SIZE/BLOCK_MASK.  Avoid this conflict by renaming
the image definitions with the IMAGE_ prefix.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-28 18:57:43 +01:00
Josef Bacik d8de39615a btrfs-progs: ioctl: move dev-replace NO_RESULT definition into replace.c
BTRFS_IOCTL_DEV_REPLACE_RESULT_NO_RESULT is defined to make sure we
differentiate internal errors from actual error codes that come back
from the device replace ioctl.  Take this out of ioctl.c and move it
into replace.c.

Though it's in public header ioctl.h, there are no users for that so it
can be moved.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-28 18:57:43 +01:00
Josef Bacik 53f8b8fd01 btrfs-progs: make btrfs_qgroup_level helper match the kernel
We return __u16 in the kernel, as this is actually the size of
btrfs_qgroup_level.  Adjust the existing helpers and update all the
callers to deal with the new size appropriately.  This will make syncing
the kernel code cleaner.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-28 18:57:43 +01:00
Josef Bacik 03451430de btrfs-progs: rename qgroup items to match the kernel naming scheme
We're going to sync the kernel source into btrfs-progs, and in the
kernel we have all these qgroup fields named with short names instead of
the full name, so rename

referenced -> rfer
compressed -> cmpr
exclusive -> excl

to match the kernel and update all the users, this will make the sync
cleaner.

ioctl.h is a public header but there are no users of the
btrfs_qgroup_limit structure.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-28 18:57:43 +01:00
Josef Bacik 557d2617bc btrfs-progs: ioctl: move btrfs_err_str into common/utils.h
This doesn't really belong with the ioctl definitions, and when we sync
the ioctl definitions with the kernel this helper will go away, so
adjust this now.

The ioctl.h is a public API header but the helper is not used in any 3rd
party tool so we can safely move it.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-28 18:57:43 +01:00
Josef Bacik 0d5767e664 btrfs-progs: build: use -std=gnu11
The kernel switched to this recently, switch btrfs-progs to this as well
to avoid issues with syncing the kernel code.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-24 22:36:56 +01:00
Josef Bacik a8531bfa39 btrfs-progs: fix make clean to clean convert properly
We were not clearing the .o files for btrfs-convert as we had the wrong
directory, which meant I missed a compile error that happened when I was
messing with kernel-shared.  Fix this by making sure we clear the .o
files for convert properly.

Fixes: 753baf2443 ("btrfs-progs: build: redirect dependency files files to .deps")
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-24 22:35:47 +01:00
Josef Bacik 66d6d56f04 btrfs-progs: build: turn on more compiler warnings and use -Wall
In converting some of our helpers to take new args I would miss some
locations because we don't stop on any warning, and I would miss the
warning in the scrollback.

Note: the Centos7 build is not yet warning-free so we can't enable
-Werror by default. It can be used as:

  make EXTRA_CFLAGS=-Werror

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
[ add note ]
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-24 22:35:16 +01:00
David Sterba ff964ba6ae
Btrfs progs v6.0.2
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-24 17:48:31 +01:00
David Sterba 7d1dba835d btrfs-progs: update CHANGES for 6.0.2
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-24 17:48:05 +01:00