Commit graph

281 commits

Author SHA1 Message Date
David Sterba
55f22f2106 btrfs-progs: use strncpy_null everywhere
Use the safe version of strncpy that makes sure the string is
terminated.

To be noted:

- the conversion in scrub path handling was skipped
- sizes of device paths in some ioctl related structures is
  BTRFS_DEVICE_PATH_NAME_MAX + 1

Recently gcc 13.3 started to detect problems with our use of strncpy
potentially lacking the null terminator, warnings like:

cmds/inspect.c: In function ‘cmd_inspect_logical_resolve’:
cmds/inspect.c:294:33: warning: ‘__builtin_strncpy’ specified bound 4096 equals destination size [-Wstringop-truncation]
  294 |                                 strncpy(mount_path, mounted, PATH_MAX);
      |                                 ^

Signed-off-by: David Sterba <dsterba@suse.com>
2024-06-20 10:15:52 +09:30
David Sterba
4748d400c4 btrfs-progs: rename and move __strncpy_null to string-utils
Now that there's only __strncpy_null we can drop the underscore and move
it to string-utils as it's a generic string function rather than
something for paths.

Signed-off-by: David Sterba <dsterba@suse.com>
2024-06-20 10:15:52 +09:30
David Sterba
84d88689c1 btrfs-progs: copy entire label buffer to target buffers
The label is of a fixed size 256 bytes and expects the zero terminator.
Using __strncpy_null is correct as it makes sure there's always the zero
termination but the argument passed in skips the last character.

Signed-off-by: David Sterba <dsterba@suse.com>
2024-06-20 10:15:52 +09:30
Qu Wenruo
1c71f70af0 btrfs-progs: move RST back to experimental
Currently raid-stripe-tree feature is still experimental as it requires
a BTRFS_DEBUG kernel to recognize it.  To avoid confusion move it back
to experimental so regular users won't incorrectly set it.

And since RST is no longer supported by default, also change the RST
profile detection so that for non-experimental build we won't enable RST
according to the data profiles.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-06-20 10:15:52 +09:30
Naohiro Aota
970c1a543c btrfs-progs: mkfs: align byte_count with sectorsize and zone size
While "byte_count" is eventually rounded down to sectorsize at make_btrfs()
or btrfs_add_to_fs_id(), it would be better round it down first and do the
size checks not to confuse the things.

Also, on a zoned device, creating a filesystem whose size is not aligned
to the zone boundary can be confusing. Round it down further to the zone
boundary.

The size calculation with a source directory is also tweaked to be aligned.
device_get_partition_size_fd_stat() must be aligned down not to exceed the
device size. And, btrfs_mkfs_size_dir() should have return sectorsize aligned
size. So, add an UASSERT for it.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-06-03 21:17:34 +02:00
Naohiro Aota
48c5ef6582 btrfs-progs: mkfs: fix minimum size calculation for zoned mode
Currently, we check if a device is larger than 5 zones to determine we can
create btrfs on the device or not. Actually, we need more zones to create
DUP block groups, so it fails with "ERROR: not enough free space to
allocate chunk". Implement proper support for non-SINGLE profile.

Also, current code does not ensure we can create tree-log BG and data
relocation BG, which are essential for the real usage. Count them as
requirement too.

The calculation for a regular btrfs is also adjusted to use dev_stripes
style.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
2024-06-03 21:17:15 +02:00
Naohiro Aota
2190d77a1c btrfs-progs: mkfs: unify zoned mode minimum size calc into btrfs_min_dev_size()
We are going to implement a better minimum size calculation for the zoned
mode. Move the current logic to btrfs_min_dev_size() and unify the size
checking path.

Also, convert "int mixed" to "bool mixed" while at it.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-06-03 21:15:26 +02:00
Naohiro Aota
5ff2e9c069 btrfs-progs: mkfs: remove duplicated device size check
test_minimum_size() already checks if each device can host the initial
block groups. There is no need to check if the first device can host the
initial system chunk again.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-06-03 21:14:34 +02:00
Naohiro Aota
d76d74c344 btrfs-progs: rename block_count to byte_count
block_count and dev_block_count are counting the size in bytes. And,
comparing them with e.g, "min_dev_size" is confusing. Rename them to
represent the unit better.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-06-03 21:12:47 +02:00
Qu Wenruo
c1d8bd82ca btrfs-progs: mkfs: skip failed mount check with --force
[BUG]
There is a bug report that, with very weird mount status, there can be
some mount source which can not be accessed:

  /path/dev/exports fs 500G 57G 444G 12% /path/dev/exports

Strace shows we can not access the above mount source:

 131065 stat("/path/dev/exports", 0x7ffed17b8e20) = -1 EACCES (Permission denied)

And lead to failed mount check:

 131065 write(2, "ERROR: ", 7)      = 7
 131065 write(2, "cannot check mount status of /de"..., 56) = 56
 131065 write(2, "\n", 1)        = 1

[CAUSE]
The mount check is based on libblkid, which gives the mount source, and
for non-btrfs mounts, we call path_is_reg_or_block_device() to check if
we even need to continue checking.

But in above case, the mount source is another fs, and we can not access
the source.

So we error out causing the check_mounted() to return error.

[FIX]
There is never any guarantee we can access the mount source, but on the
other hand, I do not want to ignore all access failure for the mount
source.

Let test_status_for_mkfs() to only skip check_mounted() error if
@force_overwrite is true.

This would still keep the old strict checks on whether the target is
already mounted, but if the end user really knows that certain mount
source do not need to be checked, they can always pass "-f" option to
skip the false alerts.

Link: https://bugzilla.suse.com/show_bug.cgi?id=1223799
Reported-by: Jiri Belka <jiri.belka@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-05-17 18:23:46 +02:00
David Sterba
7f396f5ced btrfs-progs: reorder key initializations
Use the objectid, type, offset natural order as it's more readable and
we're used to read keys like that.

Signed-off-by: David Sterba <dsterba@suse.com>
2024-04-30 21:49:15 +02:00
Qu Wenruo
9de25ff2ae btrfs-progs: mkfs: use proper zoned compatible write for bgt feature
[BUG]
There is a bug report that mkfs.btrfs can not specify block-group-tree
feature along with zoned devices:

  # mkfs.btrfs /dev/nullb0 -O block-group-tree,zoned
  btrfs-progs v6.7.1
  See https://btrfs.readthedocs.io for more information.

  Resetting device zones /dev/nullb0 (40 zones) ...
  NOTE: several default settings have changed in version 5.15, please make sure
        this does not affect your deployments:
        - DUP for metadata (-m dup)
        - enabled no-holes (-O no-holes)
        - enabled free-space-tree (-R free-space-tree)

  ERROR: error during mkfs: Invalid argument

[CAUSE]
During mkfs, we need to write all the 7 or 8 tree blocks into the
metadata zone, and since it's zoned device, we need to fulfill all the
requirement for zoned writes, including:

- All writes must be in sequential bytenr
- Buffer must be aligned to sector size

The sequential bytenr requirement is already met by the mkfs design, but
the second requirement on memory alignment is never met for metadata, as
we put the contents of a leaf in extent_buffer::data[], which is after a
lot of small members.

Thus metadata IO buffer would never be aligned to sector size (normally
4K).
And we require btrfs_pwrite() and btrfs_pread() to handle the memory
alignment for us.

However in create_block_group_tree() we didn't use btrfs_pwrite(), but
plain pwrite() call directly, which would lead to -EINVAL error due to
memory alignment problem.

[FIX]
Just call btrfs_pwrite() instead of the plain pwrite() in
create_block_group_tree().

Issue: #765
Pull-request: #767
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Reviewed-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-04-18 19:15:43 +02:00
David Sterba
3172d27fb9 btrfs-progs: mkfs: make transaction start and commit errors verbose
Use the templated error message for transaction failures, use the same
pattern assigning the ret and errno.

Signed-off-by: David Sterba <dsterba@suse.com>
2024-03-22 01:19:47 +01:00
David Sterba
a80d717db2 btrfs-progs: minor source sync with kernel 6.8
Sync a few more file on the source level with kernel 6.8.

- type cleanups
- defines and enums
- comments
- parameter updates
- error handling

Signed-off-by: David Sterba <dsterba@suse.com>
2024-03-12 21:22:56 +01:00
David Sterba
3d1a3c8c5a btrfs-progs: mkfs: print incompat and runtime features on one line
We've deprecated the -R option and runtime feature distinction, there's
only one option -O recommended so let it also print on the same line.
Incompat/compat/runtime status of a feature shall be consulted with the
documentation.

Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-13 07:15:37 +01:00
David Sterba
e406abb157 btrfs-progs: mkfs: warn if page does not match sectorsize
Be verbose about the potential compatibility problems with the
sectorsize and page size. Also print the page size on the overview.

Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-13 07:15:33 +01:00
Neal Gompa
e3232c2abb btrfs-progs: mkfs: make 4k sectorsize default
We have had working subpage support in Btrfs for many cycles now.
Generally, we do not want people creating filesystems by default
with non-4k sectorsizes since it creates portability problems.

As the subpage has stabilized it seems to be safe to do the switch.
This may still affect users that relying on the previous behaviour.

Issue: #604
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Eric Curtin <ecurtin@redhat.com>
Signed-off-by: Neal Gompa <neal@gompa.dev>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-01-18 02:37:45 +01:00
Qu Wenruo
389c959d6d btrfs-progs: implement arg_strtou64_with_suffix() with a new helper
This patch introduces a new parser helper, parse_u64_with_suffix(),
which has a better error handling, following all the parse_*()
helpers to return non-zero value for errors.

This new helper is going to replace parse_size_from_string(), which
would directly call exit(1) to stop the whole program.

Furthermore most callers of parse_size_from_string() are expecting
exit(1) for error, so that they can skip the error handling.

For those call sites, introduce a wrapper, arg_strtou64_with_suffix(),
to do that.  The only disadvantage is a little less detailed error
report for why the parse failed, but for most cases the generic error
string should be enough.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-01-18 02:14:23 +01:00
David Sterba
b1e2de452a btrfs-progs: mkfs: print zone count for each device
In zoned mode print zone count for each device, the zone size must be
the same so it's sufficient to print it in the summary.

  $ mkfs.btrfs -O zoned /dev/nullb[0-3]
  ...
  Zoned device:       yes
    Zone size:        16.00MiB
  ...
  Devices:
     ID        SIZE  ZONES  PATH
      1   512.00MiB     32  /dev/nullb0
      2   256.00MiB     16  /dev/nullb1
      3     1.00GiB     64  /dev/nullb2
      4     2.00GiB    128  /dev/nullb3

Issue: #693
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-03 18:04:37 +01:00
David Sterba
2069bfe016 btrfs-progs: mkfs: drop unsigned long long casts for printf
The %llu specifier does not need the typecast for ULL for a long time,
remove it.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-03 18:04:37 +01:00
David Sterba
b4f43d72ff btrfs-progs: mkfs: support parametric zone size
In experimental build, read global '--param zone-size=SIZE' and use it
as emulated zone size.  This is for testing only, will be promoted to a
proper option in the future.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-03 18:04:37 +01:00
David Sterba
9908894102 btrfs-progs: mkfs: validate device uuid set on command line
We need to validate the device uuid the same way as the fsid:

  $ ./mkfs.btrfs --device-uuid 18eabcf0-6766-4fbf-b366-71b4ae725b2- img
  btrfs-progs v6.5.2
  See https://btrfs.readthedocs.io for more information.

  ERROR: could not parse device UUID: 18eabcf0-6766-4fbf-b366-71b4ae725b2-

Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-21 15:51:07 +02:00
David Sterba
d8032c3b8b btrfs-progs: mkfs: print device uuid if set from command line
Print the device uuid in the summary in case it's specified on the
command line, not always as it would be confusing and is not usually
needed. Can be found in 'btrfs inspect-internal dump-super' as
device_item.uuid .

Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-21 15:51:07 +02:00
Anand Jain
3f27e60866 btrfs-progs: mkfs: add option to specify device uuid
Add option --device-uuid that will set the device uuid item in super
block.

This is useful for creating a filesystem with a specific device uuid,
namely for testing.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-21 15:51:07 +02:00
Anand Jain
b0c4dfaaac btrfs-progs: document allowing duplicate fsid
The commit ("btrfs-progs: allow duplicate fsid for single device
filesystems") lets the duplicate fsid used for a new mkfs document this.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-21 15:51:07 +02:00
Qu Wenruo
0af60b1faf btrfs-progs: mkfs: add the missing xattr for the rootdir inode
[BUG]
When using "mkfs.btrfs" with "--rootdir" option, the top level inode
(rootdir) will not get the same xattr from the source dir:

  mkdir -p source_dir/
  touch source_dir/file
  setfattr -n user.rootdir_xattr source_dir/
  setfattr -n user.regular_xattr source_dir/file
  mkfs.btrfs -f --rootdir source_dir $dev
  mount $dev $mnt
  getfattr $mnt
  # Nothing <<<
  getfattr $mnt/file
  # file: $mnt/file
  user.regular_xattr <<<

[CAUSE]
In function traverse_directory(), we only call add_xattr_item() for all
the child inodes, not really for the rootdir inode itself, leading to
the missing xattr items.

Not only xattr, in fact we also miss the uid/gid/timestamps/mode for the
rootdir inode.

[FIX]
Extract a dedicated function, copy_rootdir_inode(), to handle every
needed attributes for the rootdir inode, including:

- xattr
- uid
- gid
- mode
- timestamps

Issue: #688
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-21 15:51:07 +02:00
Qu Wenruo
4f5a455d1b btrfs-progs: mkfs: do not enlarge the target block device
[BUG]
When running mkfs.btrfs with --rootdir on a block device, and the source
directory contains a sparse file, whose size is larger than the block
size, then mkfs.btrfs would fail:

  # lsblk  /dev/test/test
  NAME      MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
  test-test 253:0    0  10G  0 lvm
  # mkdir -p /tmp/output
  # truncate -s 20G /tmp/output/file
  # mkfs.btrfs -f --rootdir /tmp/output /dev/test/test
  # sudo mkfs.btrfs  -f /dev/test/scratch1  --rootdir /tmp/output/
  btrfs-progs v6.3.3
  See https://btrfs.readthedocs.io for more information.

  ERROR: unable to zero the output file

[CAUSE]
Mkfs.btrfs would try to zero out the target file according to the total
size of the directory.

However the directory size is calculated using the file size, not the
real bytes taken by the file, thus for such sparse file with holes only,
it would still take 20G.

Then we would use that 20G size to zero out the target file, but if the
target file is a block device, we would fail as we can not enlarge a block
device.

[FIX]
When zeroing the file, we only enlarge it if the target is a regular
file.
Otherwise we warn about the size and continue.

Please note that, since "mkfs.btrfs --rootdir" doesn't handle sparse
file any differently from regular file, above case would still fail due
to ENOSPC, as will write zeros into the target file inside the fs.

Proper handling for sparse files would need a new series of patch to
address.

Issue: #653
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-21 15:51:07 +02:00
David Sterba
b421fdff95 btrfs-progs: move raid-stripe-tree and squota build out of experimental
The kernel patches for RST and squota are queued for 6.7, we need to be
able to test the features so it's not necessary to hide the mkfs support
under experimental build. The kernel may still need debug build to
enable mount.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-17 19:33:59 +02:00
David Sterba
21aa6777b2 btrfs-progs: clean up includes, using include-what-you-use
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-03 01:11:57 +02:00
Josef Bacik
0fa89a9da7 btrfs-progs: move btrfs_uuid_tree_add into mkfs/main.c
This function is only used in mkfs, and doesn't exist in the kernel in
ctree.c.  Additionally we have a uuid lookup function to see if the uuid
exists in the tree, which for mkfs it won't because we just created the
tree.  Move btrfs_uuid_tree_add into mkfs, and remove the lookup
function as it's not needed.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-03 01:11:56 +02:00
Josef Bacik
3808db2b3e btrfs-progs: move btrfs_record_file_extent and code into a new file
This function and it's related functions only exist for the utilities
that populate existing file systems, and do not exist in the upstream
kernel.  Move this function and the related function into it's own
common source file and out of the kernel-shared sources, and then update
all of the users to include the new location of this code.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-03 01:11:56 +02:00
Josef Bacik
8069b8b8cd btrfs-progs: drop btrfs_init_path
This simply zero's out the path, and this is used everywhere we use a
stack path.  Drop this usage and simply init the path's to empty instead
of using a function to do the memset.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-03 01:11:56 +02:00
Boris Burkov
14ac1a6051 btrfs-progs: mkfs: add support for squota
Add the ability to enable simple quotas from mkfs with '-O squota'

There is some complication around handling enable_gen while still
counting the root node of an fs. To handle this, employ a hack of doing
a no-op write on the root node to bump its generation up above that of
the qgroup enable generation, which results in counting it properly.

Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-03 01:11:55 +02:00
Anand Jain
ff4c4a3a00 btrfs-progs: allow duplicate fsid for single device filesystems
For single device btrfs filesystem, allow duplicate fsid to be created.
This should be used with caution as more devices with the same uuid
could be confused with each other.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-02 18:41:08 +02:00
Johannes Thumshirn
fff57d3774 btrfs-progs: load zone info for all zoned devices
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-02 18:41:08 +02:00
Johannes Thumshirn
b4ab282686 btrfs-progs: allow zoned RAID
Allow for RAID levels 0, 1 and 10 on zoned devices if the RAID stripe tree
is used.

Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-02 18:41:08 +02:00
David Sterba
76c0446bec btrfs-progs: mkfs: convert int to bool in a few helpers
Signed-off-by: David Sterba <dsterba@suse.com>
2023-07-27 14:45:29 +02:00
Anand Jain
d46a0ef6a0 btrfs-progs: rename struct open_ctree_flags to open_ctree_args
The struct open_ctree_flags currently holds arguments for
open_ctree_fs_info(), it can be confusing when mixed with a local variable
named open_ctree_flags as below in the function cmd_inspect_dump_tree().

  cmd_inspect_dump_tree()
  ::
  struct open_ctree_flags ocf = { 0 };
  ::
  unsigned open_ctree_flags;

So rename struct open_ctree_flags to struct open_ctree_args.

Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-07-26 15:00:47 +02:00
Dominique Martinet
9362803539 btrfs-progs: mkfs: make --quiet silence the 5.15 default change NOTE
mkfs.btrfs help message for --quiet is 'no message except errors' so
we probably ought to silence this as well in the quiet case.

Author: Dominique Martinet <dominique.martinet@atmark-techno.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-06-09 11:57:39 +02:00
David Sterba
ae73e89f28 btrfs-progs: mkfs: more verbose output for --rootdir
Print the source directory for --rootdir and if --shrink is used. With
-vv then print the individual files as added:

  $ mkfs.btrfs --rootdir dir --shrink -vv img
  ...
  Rootdir from:       Documentation
  ADD: /btrfs-progs/Documentation/btrfs-check.rst
  ...
  ADD: /btrfs-progs/Documentation/btrfs-send.rst
    Shrink:           yes
  Label:              (null)
  UUID:               40d3a16f-02d8-40d7-824b-239cee528093
  ...

The 'Rootdir from' is printed before the files are added so there's now
message before the files are added which could take some time.

Issue: #627
Signed-off-by: David Sterba <dsterba@suse.com>
2023-05-26 22:17:33 +02:00
David Sterba
95c1fa1871 btrfs-progs: mkfs: remove redundant variable for source dir
Validity of source dir can be determined by the variable itself, no need
to track it separately.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-05-26 22:17:33 +02:00
Qu Wenruo
08a3bd7694 btrfs-progs: tune: add the ability to generate new data checksums
This patch would modify btrfs_csum_file_block() to handle csum type
other than the one used in the current fs.

The new data checksum would use a different objectid (-13) to
distinguish with the existing one (-10).
This needs to change tree-checker to skip the item size checks,
since new csum can be larger than the original csum.

After this stage, the resulted csum tree would look like this:

	item 0 key (CSUM_CHANGE EXTENT_CSUM 13631488) itemoff 8091 itemsize 8192
		range start 13631488 end 22020096 length 8388608
	item 1 key (EXTENT_CSUM EXTENT_CSUM 13631488) itemoff 7067 itemsize 1024
		range start 13631488 end 14680064 length 1048576

Note the itemsize is 8 times the original one, as the original csum is
CRC32, while target csum is SHA256, which is 8 times the size.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-05-26 18:02:32 +02:00
Qu Wenruo
46364d3766 btrfs-progs: replace write_and_map_eb() by write_data_to_disk()
The function write_and_map_eb() is quite abused as a way to write any
generic buffer back to disk.

But we have a more suitable function already, write_data_to_disk().

This patch would remove the abused write_data_to_disk() calls, and
convert the only three valid call sites to write_data_to_disk() instead.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-05-26 18:02:31 +02:00
Josef Bacik
f8efe9f724 btrfs-progs: sync file-item.h into progs
This patch syncs file-item.h into btrfs-progs.  This carries with it an
API change for btrfs_del_csums, which takes a root argument in the
kernel, so all callsites have been updated accordingly.

I didn't sync file-item.c because it carries with it a bunch of bio
related helpers which are difficult to adapt to the kernel.
Additionally there's a few helpers in the local copy of file-item.c that
aren't in the kernel that are required for different tools.

This requires more cleanups in both the kernel and progs in order to
sync file-item.c, so for now just do file-item.h in order to pull things
out of ctree.h.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-05-26 18:02:29 +02:00
Josef Bacik
c979ffd787 btrfs-progs: sync accessors.[ch] from the kernel
This syncs accessors.[ch] from the kernel.  For the most part
accessors.h will remain the same, there's just some helpers that need to
be adjusted for eb->data instead of eb->pages.  Additionally accessors.c
needed to be completely updated to deal with this as well.

This is a set of files where we will likely only sync the header going
forward, and leave the C file in place as it needs to be specific to
btrfs-progs.

This forced a few "unrelated" changes

- Using btrfs_dir_item_ftype() instead of btrfs_dir_item_type().  This
  is due to the encryption changes, and was simpler to just do in this
  patch.
- Adjusting some of the print tree code to use the actual helpers and
  not the btrfs-progs ones.

A local definition of static_assert is used to avoid compilation
failures on older gcc (< 9) where the 2nd parameter is mandatory.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-05-26 18:02:28 +02:00
Josef Bacik
a754fe29d9 btrfs-progs: sync uapi/btrfs.h into btrfs-progs
We want to keep this file locally as we want to be uptodate with
upstream, so we can build btrfs-progs regardless of which kernel is
currently installed.  Sync this with the upstream version and put it in
kernel-shared/uapi to maintain some semblance of where this file comes
from.

There are some changes that need to be synced back to kernel. A local
definition of static_assert is used to avoid compilation problems on gcc
(< 9) due to mandatory 2nd parameter.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-05-26 18:02:28 +02:00
Josef Bacik
bf0f3db765 btrfs-progs: introduce UASSERT() for purely userspace code
While syncing messages.[ch] I had to back out the ASSERT() code in
kerncompat.h, which means we now rely on the kernel code for ASSERT().
In order to maintain some semblance of separation introduce UASSERT()
and use that in all the purely userspace code.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-05-26 18:02:28 +02:00
psykose
c9abbf6264 btrfs-progs: stop using legacy *64 interfaces
The *64 interfaces, such as fstat64, off64_t, etc, are legacy interfaces
created at a time when 64-bit file support was still new. They are
generally exposed when defining a macro named _LARGEFILE64_SOURCE, as
e.g. the glibc docs[0] say.

The modern way to utilise largefile support, is to continue to use the
regular interfaces (off_t, fstat, ..), and define _FILE_OFFSET_BITS=64.

We already use the autoconf macro AC_SYS_LARGEFILE[1] which arranges this
and sets this macro for us. Therefore, we can utilise the non-64 names
without fear of breaking on 32-bit systems.

This fixes the build against musl libc, ever since musl dropped the
*64 compat from interfaces by default[2] just for _GNU_SOURCE, unless
_LARGEFILE64_SOURCE is defined. However, there are plans for a future
removal of the whole *64 header API, and that workaround (adding another
define) might cease to exist.

So, rename all *64 API use to the regular non-suffixed names. For
consistency, rename the internal functions that were *64 named
(lstat64_path, ..) too.

This should have no regressions on any platform.

[0]: https://www.gnu.org/software/libc/manual/html_node/Feature-Test-Macros.html#index-_005fLARGEFILE64_005fSOURCE
[1]: https://www.gnu.org/software/autoconf/manual/autoconf-2.67/html_node/System-Services.html
[2]: 25e6fee27f

Pull-request: #615
Signed-off-by: psykose <alice@ayaya.dev>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-04-25 16:59:42 +02:00
Qu Wenruo
b2a1be83b8 btrfs-progs: mkfs: keep file descriptors open during whole time
[BUG]
There is an internal bug report that, after mkfs.btrfs there is a chance
that no /dev/disk/by-uuid/<uuid> symlink is not created at all.

[CAUSE]
That uuid symlink is created by udev, which listens to inotify
IN_CLOSE_WRITE events from all block devices.

After such IN_CLOSE_WRITE event is triggered, udev would *disable*
inotify for that block device, and do a blkid scan on it.
After the blkid scan is done, re-enables the inotify listening.

This means normally mkfs tools should open the fd, do all the writes,
and close the fd after everything is done.

But unfortunately for mkfs.btrfs, it's not the case, we have a lot of
phases separated by different close() calls:

  open_ctree() would open fds of each involved device
  and close them at close_ctree()
  Only after close_ctree() we have a valid superblock -\
                                                       |
|<------- A -------->|<--------- B --------->|<------- C ------->|
          |                      |
          |                      `- open a new fd for make_btrfs()
          |                         and close it before open_ctree()
          |                         The device contains invalid sb.
          |
          `- open a new fd for each device, then call
             btrfs_prepare_device(), then close the fd.
             The device would contain no valid superblock.

If at the close() of phase A udev event is triggered, while doing udev
scan we go into phase C (but before the new valid super blocks written),
udev would only see no superblock or invalid superblock.

Then phase C finished, udev resumes its inotify listening, but at this
time mkfs is finished, while udev only sees the premature data from
phase A, and misses the IN_CLOSE_WRITE events from phase C.

[FIX]
Instead of opening and closing a new fd for each device, re-use the fd
opened during prepare_one_device(), and close all the fds until
close_ctree() is called.

By this, although we may still have race between close_ctree() and
explicit close() calls, at least udev can always see the properly
written super blocks.

To compensate the change, some extra cleanups are made:

- Do not touch @device_count
  Which makes later prepare_ctx iteration much easier.

- Remove top-level @fd variable
  Instead go with prepare_ctx[i].fd.

- Do not open with O_RDWR in test_dev_for_mkfs()
  as test_dev_for_mkfs() would close the fd, if we go O_RDWR, it can
  cause the udev race.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-04-25 16:59:41 +02:00
Qu Wenruo
4dbe66ca2f btrfs-progs: mkfs: make -R|--runtime-features option deprecated
The option -R|--runtime-features was introduced to support features that
don't result in a full incompat flag change, thus things like
free-space-tree and quota features are put here.

But to end users, such separation of features is not helpful and can be
sometimes confusing.

Thus we're already migrating those runtime features into -O|--features
option under experimental builds.

I believe this is the proper time to move those runtime features into
-O|--features option, and mark the -R|--runtime-features option
deprecated.

For now we still keep the old option as for compatibility purposes.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-04-17 19:27:53 +02:00