Commit graph

5176 commits

Author SHA1 Message Date
Sheng Mao 4498fe1a2a libbtrfsutil: add pkg-config spec file
Add pkg-config file for libbtrfsutil which can be used to set up
compilation and link flags.

The paths in pc file depend on prefix variable but ignore DESTDIR.
DESTDIR is used for packaging and it should not affect the paths in pc
file.

Issue: #285
Signed-off-by: Sheng Mao <shngmao@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-20 15:49:07 +02:00
David Sterba f43df8a863 btrfs-progs: ci: add sanitizer coverage
Add builds with ASAN and UBSAN for both gcc. Clang + asan fails at
some tests due to wrong asan runtime, clang + ubsan fails at
libbtrfsutil/btrfsutil_internal.h:71:20:

	memcpy(&tmp, &sh->type, sizeof(__u32));

which seems to be too strict.

    [TEST/cli]   008-subvolume-get-set-default
subvolume get-default: default id is not 256, but
libbtrfsutil/btrfsutil_internal.h:71:20: runtime error: member access within
misaligned address 0x7ffcc8017f67 for type 'const struct
btrfs_ioctl_search_header', which requires 8 byte alignment

0x7ffcc8017f67: note: pointer points here

 00 00 00 00 07  00 00 00 00 00 00 00 00  01 00 00 00 00 00 00 05  00 00 00 00 00 00 00 90  00 00 00
             ^

libbtrfsutil/btrfsutil_internal.h:64:20: runtime error: member access within
misaligned address 0x7ffcc8017f67 for type 'const struct
btrfs_ioctl_search_header', which requires 8 byte alignment

0x7ffcc8017f67: note: pointer points here

 00 00 00 00 07  00 00 00 00 00 00 00 00  01 00 00 00 00 00 00 05  00 00 00 00 00 00 00 90  00 00 00
             ^

libbtrfsutil/btrfsutil_internal.h:78:20: runtime error: member access within
misaligned address 0x7ffcc8017f67 for type 'const struct
btrfs_ioctl_search_header', which requires 8 byte alignment

0x7ffcc8017f67: note: pointer points here

 00 00 00 00 07  00 00 00 00 00 00 00 00  01 00 00 00 00 00 00 05  00 00 00 00 00 00 00 90  00 00 00

Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-20 15:49:07 +02:00
David Sterba 726026f7da btrfs-progs: build: print versions for all subprojects in summary
Update the summary after configure finishes with versions of all
projects that are now hosted in this repository. Mark libbtrfs as
deprecated do discourage new use, formal deprecation will happen in the
future.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-20 15:49:07 +02:00
David Sterba 35614d8a64 btrfs-progs: sb-mod: add syntax to the help text
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-20 15:49:07 +02:00
David Sterba bd017ce732 btrfs-progs: sb-mod: add remaining dev_item members
Add helpers and all remaining members of the device item embedded in the
superblock. No UUID types.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-20 15:49:07 +02:00
David Sterba 8de6816b47 btrfs-progs: sb-mod: add dev_item prefix for sb::device members
For parity with output of 'btrfs inspect dump-super', add the
"dev_item." prefix to the devid member. This allows adding more
members without conflicts (like it would be for device total_bytes).

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-20 15:49:07 +02:00
Anand Jain c875145cb3 btrfs-progs: sb-mod: add devid to the modifiable list
We need this patch to create a crafted image with bogus devid.

For example:
./btrfs-sb-mod devid =0

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-20 15:49:07 +02:00
Sidong Yang 4179885cf9 btrfs-progs: docs: add qgroup examples
This patch adds examples section in btrfs-qgroup documentation.
The example make a quota group that has two subvolumes as children.

Signed-off-by: Sidong Yang <realwakka@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-02 20:12:09 +02:00
Rafostar 70411ea81b btrfs-progs: scrub: make scrub status summary easier to read
Currently most btrfs commands separate their output with empty lines
which makes them more human readable. The scrub cmd when used with -d
arg to show per device information does not. It makes it harder to find
values for current disk because they are not separated from each other.
This commit adds an empty line after each device summary to make it
match other btrfs cmd outputs.

For some reason this was the only line in scrub status that did not
start from capital letter. Now it is more consistent with the rest.

Pull-request: #256
Author: Rafostar <Rafostar@users.noreply.github.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-02 20:12:09 +02:00
Forza-tng fbe95d714f btrfs-progs: docs: update limits for minimal device size
Add limits:

  # mkfs.btrfs loop.img
  btrfs-progs v5.7
  See http://btrfs.wiki.kernel.org for more information.

  ERROR: 'loop.img' is too small to make a usable filesystem
  ERROR: minimum size for each btrfs device is 114294784

  # mkfs.btrfs --mixed loop.img
  btrfs-progs v5.7
  See http://btrfs.wiki.kernel.org for more information.

  ERROR: 'loop.img' is too small to make a usable filesystem
  ERROR: minimum size for each btrfs device is 16777216

Pull-request: #287
Author: Forza-tng <Forza-tng@users.noreply.github.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-02 20:12:09 +02:00
David Sterba f1b428ea3b btrfs-progs: tests: add case for send vs subvolume deletion
Subvolumes that are part of send must not be deleted, we can check that
only by the error code EPERM, otherwise it's also in the system log.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-02 20:12:08 +02:00
David Sterba 74f45db76c btrfs-progs: subvolume delete: print message for EPERM, possible send in progress
If subvolume deletion fails with EPERM, the most common reasons are that
it's a default subvolume (addressed by an earlier patch) or that the
subvolume is part of a send operation. This is printed to the system log
and there's no information available for user space, but at least the
warning can hint the user that something could be going on.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-02 20:12:08 +02:00
David Sterba 8b4f9546c3 btrfs-progs: tests: add case for deleting default subvolume
Deleting the default subvolume must not succeed and must print a
message to terminal.

Issue: #274
Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-02 20:12:08 +02:00
Sidong Yang 87804a3f06 btrfs-progs: subvolume: check deleting default subvolume
Deleting the default subvolume is not permitted and kernel prints a
message to the system log. This is not immediately clear to the user and
we had requests to improve that.

This patch will read the default subvolume id and reject deletion
without trying to delete it.

Issue: #274
Issue: #255
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=207975
Signed-off-by: Sidong Yang <realwakka@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-02 20:04:08 +02:00
Marcos Paulo de Souza 65ecbc7e0e btrfs-progs: convert: show more info when reserve_space fails
btrfs-convert currently can't handle more fragmented block groups when
converting ext4 because the minimum size of a data chunk is 32MiB.

When converting an ext4 fs with more fragmented block group with the disk
almost full, we can end up hitting a ENOSPC problem [1] since smaller
block groups (10MiB for example) end up being extended to 32MiB, leaving
the free space tree smaller when converting it to btrfs.

This patch adds error messages telling which needed bytes couldn't be
allocated from the free space tree and shows the largest portion available:

  create btrfs filesystem:
	  blocksize: 4096
	  nodesize:  16384
	  features:  extref, skinny-metadata (default)
	  checksum:  crc32c
  free space report:
	  total:     1073741824
	  free:      39124992 (3.64%)
  ERROR: failed to reserve 33554432 bytes for metadata chunk, largest available: 33488896 bytes
  ERROR: unable to create initial ctree: No space left on device

Issue: #251
Reviewed-by: Neal Gompa <ngompa13@gmail.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-09-29 22:46:12 +02:00
Boris Burkov 92d92e99b7 btrfs-progs: mkfs: support free space tree as -R option
Add a runtime feature (-R) flag for the free space tree. A filesystem
that is mkfs'd with -R free-space-tree then mounted with no options has
the same contents as one mkfs'd without the option, then mounted with
'-o space_cache=v2'.

The only tricky thing is in exactly how to call the tree creation code.
Using btrfs_create_free_space_tree as is did not quite work, because an
extra reference to the eb (root->commit_root) is leaked, which mkfs
complains about with a warning. I opted to follow how the uuid tree is
created by adding it to the dirty roots list for cleanup by
commit_tree_roots in commit_transaction. As a result,
btrfs_create_free_space_tree no longer exactly matches the version in
the kernel sources.

Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-09-08 22:06:04 +02:00
David Sterba 5011ef94ac btrfs-progs: ci: install clang unconditionally
The builds on travis started to fail for the clang and it was missing
completely, so add it to the list of packages.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-09-08 21:59:52 +02:00
gandalf3 07fd775b13 btrfs-progs: docs: fix typo in btrfs restore message
mirros -> mirrors

Pull-request: #284
Author: gandalf3 <gandalf3@users.noreply.github.com>
Reviewed-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-09-08 21:59:52 +02:00
David Sterba cfe26cca7a btrfs-progs: ci: enable free-space-tree for all mkfs tests
Run all mkfs tests with -R free-space-tree in case the kernel supports
that (4.5+ required).

Signed-off-by: David Sterba <dsterba@suse.com>
2020-09-08 21:59:52 +02:00
Nikolay Borisov c08983291e btrfs-progs: check: fix error reporting on root inode
If btrfs check detects an error on the root inode of a subvolume it
prints:

    Opening filesystem to check...
    Checking filesystem on /dev/vdc
    UUID: 4ac7a216-bf97-4c5f-9899-0f203c20d8af
    [1/7] checking root items
    [2/7] checking extents
    [3/7] checking free space cache
    [4/7] checking fs roots
    root 5 root dir 256 error
    ERROR: errors found in fs roots
    found 196608 bytes used, error(s) found
    total csum bytes: 0
    total tree bytes: 131072
    total fs tree bytes: 32768
    total extent tree bytes: 16384
    btree space waste bytes: 124376
    file data blocks allocated: 65536
     referenced 65536

This is not very helpful since there is no specific information about
the exact error. This is due to the fact that check_root_dir doesn't
set inode_record::errors accordingly. This patch rectifies this and now
the output would look like:

	[1/7] checking root items
	[2/7] checking extents
	[3/7] checking free space cache
	[4/7] checking fs roots
	root 5 inode 256 errors 2000, link count wrong
	ERROR: errors found in fs roots
	found 196608 bytes used, error(s) found
	total csum bytes: 0
	total tree bytes: 131072
	total fs tree bytes: 32768
	total extent tree bytes: 16384
	btree space waste bytes: 124376
	file data blocks allocated: 65536
	 referenced 65536

Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:16:36 +02:00
Qu Wenruo fedc94091f btrfs-progs: tests: add test image to fsck/043 for inode transid repair
The image would contain an inode with invalid transid:

        item 4 key (257 INODE_ITEM 0) itemoff 15881 itemsize 160
                generation 6 transid 134217734 size 131072 nbytes 131072
                block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0

Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:09:55 +02:00
Qu Wenruo 0eafacf668 btrfs-progs: check/original: add inode transid detect and repair support
The inode transid detect and repair is reusing the existing inode
geneartion code.

Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:09:55 +02:00
Qu Wenruo ce2dcb3615 btrfs-progs: check/lowmem: add inode transid detect and repair support
There are quite some reports on kernel rejecting invalid inode
generation, but it turns out to be that, kernel is just rejecting inode
transid. It's a bug in kernel error message.

To solve the problem and make the fs mountable again, add the detect and
repair support for lowmem mode.

The implementation is pretty much the same, just re-use the existing
inode generation detect and repair code.

Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:09:55 +02:00
Marcos Paulo de Souza c655b5e4b1 btrfs-progs: make btrfs_lookup_dir_index in parity with kernel code
This function exists in kernel side but using the _item suffix, and
objectid argument is placed before the name argument. Change the
function to reflect the kernel version.

Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:09:49 +02:00
David Sterba e05111d11e btrfs-progs: ci: fix libreiserfs build
The CI build fails with:

../include/misc.h:76:26: error: unknown type name ‘loff_t’; did you mean ‘off_t’?
   76 | int valid_offset(int fd, loff_t offset);
      |                          ^~~~~~
      |                          off_t

Add the right define that will bring the symbol.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:06 +02:00
David Sterba 09252e76ae btrfs-progs: ci: updated zstd version to 1.4.5
There's a new major version of zstd, without any obvious changes that
would affect our build testing coverage.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:06 +02:00
David Sterba 692426265a btrfs-progs: merge find-root.c and btrfs-find-root.c
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:06 +02:00
David Sterba 0144bcb713 btrfs-progs: move volumes.c to kernel-shared/
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:06 +02:00
David Sterba 6069bc52a9 btrfs-progs: move transaction.c to kernel-shared/
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:06 +02:00
David Sterba 978f300c21 btrfs-progs: move inode.c to kernel-shared/
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:05 +02:00
David Sterba abb670f883 btrfs-progs: move ctree.c to kernel-shared/
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:05 +02:00
David Sterba c03619b864 btrfs-progs: move file.c to kernel-shared/
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:05 +02:00
David Sterba 772f0da6df btrfs-progs: move disk-io.c to kernel-shared/
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:05 +02:00
David Sterba cf529f36ad btrfs-progs: move print-tree.c to kernel-shared/
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:05 +02:00
David Sterba 7dd4abc3c5 btrfs-progs: move extent-tree.c to kernel-shared/
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:04 +02:00
David Sterba 4e49bd703d btrfs-progs: move extent_io.c to kernel-shared/
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:04 +02:00
David Sterba da90f38ad9 btrfs-progs: move free-space-tree.c to kernel-shared/
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:04 +02:00
David Sterba cdaf906d68 btrfs-progs: move send-utils.c to common/
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:04 +02:00
David Sterba f6009c4cb3 btrfs-progs: move send-stream.c to common/
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:04 +02:00
David Sterba e2995c1cce btrfs-progs: move send-dump.c to cmds/receive-dump.c
The dump is a command mode of receive.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:04 +02:00
David Sterba 687ca4d127 btrfs-progs: move utils-lib.c to common/
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:04 +02:00
David Sterba a4122790ac btrfs-progs: move extent-cache.c to common/
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:04 +02:00
David Sterba 9f456949f3 btrfs-progs: add flat include switch to send.h
The send.h is exported for libbtrfs and includes ctree.h but it lacks
the in-tree/out-of-tree ifdef switch. This would be a problem once
ctree.h moves.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:04 +02:00
David Sterba 33667904dc btrfs-progs: tests: skip cli-tests/014 in travis
The test fails inside travis, check for a multiple-profile detection.
This was verified to work on a proper host so the workaround is in place
let the whole suite pass.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:03 +02:00
Qu Wenruo 127973d723 btrfs-progs: remove the unused variable in check_chunks_and_extents_lowmem()
The variable @root is only set but not utilized, while we only utilize
@root1.

Replace @root1 with @root, then remove the @root1.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:03 +02:00
David Sterba 2335cd02b3 btrfs-progs: check: drop unused fs_info
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:03 +02:00
David Sterba fd73539253 btrfs-progs: check: replace local fs_info by global fs_info in mode-lowmem.c
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:03 +02:00
David Sterba c9c94a0f87 btrfs-progs: check: replace local fs_info by global fs_info in mode-common.c
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:03 +02:00
David Sterba 3b86c9fd82 btrfs-progs: check: replace local fs_info by global fs_info in main.c
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:03 +02:00
David Sterba cfe94be618 btrfs-progs: check: rename global_info
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:03 +02:00