Commit graph

49 commits

Author SHA1 Message Date
David Sterba a35409328e btrfs-progs: use strncpy_null everywhere
Use the safe version of strncpy that makes sure the string is
terminated.

To be noted:

- the conversion in scrub path handling was skipped
- sizes of device paths in some ioctl related structures is
  BTRFS_DEVICE_PATH_NAME_MAX + 1

Recently gcc 13.3 started to detect problems with our use of strncpy
potentially lacking the null terminator, warnings like:

cmds/inspect.c: In function ‘cmd_inspect_logical_resolve’:
cmds/inspect.c:294:33: warning: ‘__builtin_strncpy’ specified bound 4096 equals destination size [-Wstringop-truncation]
  294 |                                 strncpy(mount_path, mounted, PATH_MAX);
      |                                 ^

Signed-off-by: David Sterba <dsterba@suse.com>
2024-06-17 21:53:04 +02:00
Qu Wenruo c23e068aaf btrfs-progs: convert: rework file extent iteration to handle unwritten extents
[BUG]
There is a bug report that btrfs-convert cannot handle unwritten extents
(EXT2_EXTENT_FLAGS_UNINIT set, which is pretty much the same as
BTRFS_FILE_EXTENT_PREALLOC), which can cause the converted image to have
incorrect contents.

[CAUSE]
Currently we use ext2fs_block_iterate2() to go through all data extents
of an ext2 inode, but it doesn't provide the info on if the range is
unwritten or not.

Thus for unwritten extents, the results btrfs would just treat it as
regular extents, and read the contents from disk other than setting the
contents to zero.

[FIX]
Instead of the ext2fs_block_iterate2(), here we follow the debugfs'
"dump_extents" command, to use ext2fs_extent_*() helpers to go through
every data extent of the inode, that's if the inode supports the
EXT4_EXTENTS_FL flag.

Now we can properly get the info of which extents are unwritten, and use
holes to replace those unwritten extents.

Reported-by: Yordan <y16267966@gmail.com>
Link: https://lore.kernel.org/all/d34c7d77a7f00c93bea6a4d6e83c7caf.mailbg@mail.bg/
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-06-03 22:10:22 +02:00
David Sterba bcb887a4de btrfs-progs: convert: handle rename of inode_includes() from e2fsprogs 1.47.1
There is a new release candidate for e2fsprogs https://github.com/tytso/e2fsprogs/releases/tag/v1.47.1-rc2

Linking btrfs-progs v6.8 against this version of e2fsprogs leads to the following compile error:

convert/source-ext2.c: In function 'ext4_copy_inode_timespec_extra':
convert/source-ext2.c:733:13: warning: implicit declaration of function 'inode_includes' [-Wimplicit-function-declaration]
  733 |         if (inode_includes(inode_size, i_ ## xtime ## _extra)) {                        \
      |             ^~~~~~~~~~~~~~
convert/source-ext2.c:769:9: note: in expansion of macro 'EXT4_COPY_XTIME'
  769 |         EXT4_COPY_XTIME(atime, dst, tv_sec, tv_nsec);
      |         ^~~~~~~~~~~~~~~
convert/source-ext2.c:733:40: error: 'i_atime_extra' undeclared (first use in this function)
  733 |         if (inode_includes(inode_size, i_ ## xtime ## _extra)) {                        \
      |                                        ^~
convert/source-ext2.c:769:9: note: in expansion of macro 'EXT4_COPY_XTIME'
  769 |         EXT4_COPY_XTIME(atime, dst, tv_sec, tv_nsec);
      |         ^~~~~~~~~~~~~~~
convert/source-ext2.c:733:40: note: each undeclared identifier is reported only once for each function it appears in
  733 |         if (inode_includes(inode_size, i_ ## xtime ## _extra)) {                        \
      |                                        ^~
convert/source-ext2.c:769:9: note: in expansion of macro 'EXT4_COPY_XTIME'
  769 |         EXT4_COPY_XTIME(atime, dst, tv_sec, tv_nsec);
      |         ^~~~~~~~~~~~~~~
convert/source-ext2.c:733:40: error: 'i_mtime_extra' undeclared (first use in this function)
  733 |         if (inode_includes(inode_size, i_ ## xtime ## _extra)) {                        \
      |                                        ^~
convert/source-ext2.c:770:9: note: in expansion of macro 'EXT4_COPY_XTIME'
  770 |         EXT4_COPY_XTIME(mtime, dst, tv_sec, tv_nsec);
      |         ^~~~~~~~~~~~~~~
convert/source-ext2.c:733:40: error: 'i_ctime_extra' undeclared (first use in this function)
  733 |         if (inode_includes(inode_size, i_ ## xtime ## _extra)) {                        \
      |                                        ^~
convert/source-ext2.c:771:9: note: in expansion of macro 'EXT4_COPY_XTIME'
  771 |         EXT4_COPY_XTIME(ctime, dst, tv_sec, tv_nsec);
      |         ^~~~~~~~~~~~~~~
convert/source-ext2.c:774:40: error: 'i_crtime_extra' undeclared (first use in this function)
  774 |         if (inode_includes(inode_size, i_crtime_extra)) {
      |                                        ^~~~~~~~~~~~~~

from tytso/e2fsprogs@ca8bc92

    Fix inode_includes() macro to properly wrap "inode" parameter,
    and rename to ext2fs_inode_includes() to avoid potential name
    clashes. Use this to check inode field inclusion in debugfs
    instead of bare constants for inode field offsets.

To fix that use the new prefixed macro and add backward compatibility that
would still use inode_includes().

Issue: #785
Signed-off-by: David Sterba <dsterba@suse.com>
2024-05-03 14:53:52 +02:00
Qu Wenruo 517ba2d9e5 btrfs-progs: convert: insert a dummy inode item before inode ref for ext2/4
[BUG]
There is a report about failed btrfs-convert, which shows the following
error:

  Create btrfs metadata
  corrupt leaf: root=5 block=5001931145216 slot=1 ino=89911763, invalid previous key objectid, have 89911762 expect 89911763
  leaf 5001931145216 items 336 free space 7 generation 90 owner FS_TREE
  leaf 5001931145216 flags 0x1(WRITTEN) backref revision 1
  fs uuid 8b69f018-37c3-4b30-b859-42ccfcbe2449
  chunk uuid 448ce78c-ea41-49f6-99dc-46ad80b93da9
          item 0 key (89911762 INODE_REF 3858733) itemoff 16222 itemsize 61
                  index 171 namelen 51 name: [FILENAME1]
          item 1 key (89911763 INODE_REF 3858733) itemoff 16161 itemsize 61
                  index 103 namelen 51 name: [FILENAME2]

[CAUSE]
When iterating a directory, btrfs-convert would insert the DIR_ITEMs,
along with the INODE_REF of that inode.

This leads to above stray INODE_REFs, and trigger the tree-checker.

This can only happen for large fs, as for most cases we have all these
modified tree blocks cached, thus tree-checker won't be triggered.
But when the tree block cache is not hit, and we have to read from disk,
then such behavior can lead to above tree-checker error.

[FIX]
Insert a dummy INODE_ITEM for the INODE_REF first, the inode items would
be updated when iterating the child inode of the directory.

Issue: #731
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-01-17 01:47:39 +01:00
Qu Wenruo 631ee66639 btrfs-progs: convert: for ext2, fix possible tree-checker error when converting a large fs
[BUG]
There is a report about failed btrfs-convert, which shows the following
error:

  corrupt leaf: root=5 block=5001928998912 slot=1 ino=89911763, invalid previous key objectid, have 89911762 expect 89911763
  ERROR: failed to copy ext2 inode 89911320: -5
  ERROR: error during copy_inodes -5
  WARNING: error during conversion, the original filesystem is not modified

[CAUSE]
Above error is triggered when checking the following items inside a
subvolume:

- inode ref
- dir item/index
- file extent
- xattr

This is to make sure these items have correct previous key.

However btrfs-convert is not following this requirement, it always
inserts those items first, then creates a btrfs_inode for it.

Thus it can lead to the error.

This can only happen for large fs, as for most cases we have all these
modified tree blocks cached, thus tree-checker won't be triggered.
But when the tree block cache is not hit, and we have to read from disk,
then such behavior can lead to above tree-checker error.

[FIX]
Make sure we insert the inode item first, then the file extents/dir
items/xattrs.  And after the file extents/dir items/xattrs inserted, we
update the existing inode (to update its size and bytes).

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-01-12 16:36:36 +01:00
Josef Bacik f8efe9f724 btrfs-progs: sync file-item.h into progs
This patch syncs file-item.h into btrfs-progs.  This carries with it an
API change for btrfs_del_csums, which takes a root argument in the
kernel, so all callsites have been updated accordingly.

I didn't sync file-item.c because it carries with it a bunch of bio
related helpers which are difficult to adapt to the kernel.
Additionally there's a few helpers in the local copy of file-item.c that
aren't in the kernel that are required for different tools.

This requires more cleanups in both the kernel and progs in order to
sync file-item.c, so for now just do file-item.h in order to pull things
out of ctree.h.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-05-26 18:02:29 +02:00
Qu Wenruo f3353a5e85 btrfs-progs: convert: handle ext4 orphan file feature properly
[BUG]
Since e2fsprog 1.47, even with a newly created empty ext4 filesystem,
btrfs-convert would result an fs that btrfs-check would complain:

  # mkfs.ext4 -F test.img
  # btrfs-convert test.img
  # btrfs-check test.img
  Opening filesystem to check...
  Checking filesystem on test.img
  UUID: e45da158-8967-4e4d-9c9f-66b0d127dbce
  [1/7] checking root items
  [2/7] checking extents
  [3/7] checking free space cache
  [4/7] checking fs roots
  root 5 inode 266 errors 2000, link count wrong
  ERROR: errors found in fs roots
  found 26333184 bytes used, error(s) found <<<
  total csum bytes: 25540
  total tree bytes: 180224
  total fs tree bytes: 49152
  total extent tree bytes: 16384
  btree space waste bytes: 145423
  file data blocks allocated: 33947648
   referenced 26284032

[CAUSE]
Ext4 has a new compat feature, COMPAT_ORPHAN_FILE, as a better way to
track all the orphan inodes.

This new feature would create a new special inode for this purpose, and
such orphan file inode would not be reachable from any other inode, but
only from super block.

Unfortunately btrfs-convert only skip ext2 known special inodes, not the
newer one.

[FIX]
According to the kernel document, we can locate the orphan file inode
using ext2 super block s_orphan_file_inum, and skip it for
btrfs-convert.

And such skip would only happen if we have the definition of
EXT4_FEATURE_COMPAT_ORPHAN_FILE, to be compatible with older e2fsprogs.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-03-23 19:48:26 +01:00
David Sterba 2b909aa5db btrfs-progs: convert: do proper error handling
There are cases where the BUG_ON should be replaced by error
handling as it's validating the data from the source filesystem or
possibility to convert. The unconverted cases are asserts and will be
replaced later.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:10 +02:00
David Sterba a827bb2db8 btrfs-progs: use template for transaction commit error messages
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:10 +02:00
David Sterba 8fcafae04a btrfs-progs: use template for transaction start error messages
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:10 +02:00
David Sterba b610fc1612 btrfs-progs: convert: use message helpers for error messages
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:07 +02:00
David Sterba ec55de0ac5 btrfs-progs: convert: update include lists
The tool IWYU (include what you use) suggests to remove and add some
includes.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:12 +02:00
Nikolay Borisov 5110ad88cb btrfs-progs: convert: properly work with large ext4 filesystems
On large (blockcount > 32bit) filesystems reading directly
super_block->s_blocks_count is not sufficient as the block count is held
in 2 separate 32 bit variables. Instead always use the provided
ext2fs_blocks_count to read the value. This can result in assertion
failure, when the block count is only held in the high 32 bits, in this
case s_block_counts would be zero, which would result in
btrfs_convert_context::block_count/total_bytes to also be 0 and hit an
assertion failure:

    convert/main.c:1162: do_convert: Assertion `cctx.total_bytes != 0` failed, value 0
    btrfs-convert(+0xffb0)[0x557defdabfb0]
    btrfs-convert(main+0x6c5)[0x557defdaa125]
    /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea)[0x7f66e1f8bd0a]
    btrfs-convert(_start+0x2a)[0x557defdab52a]
    Aborted

What's worse it can also result in btrfs-convert mistakenly thinking
that a filesystem is smaller than it actually is (ignoring the top 32 bits).

Link: https://lore.kernel.org/linux-btrfs/023b5ca9-0610-231b-fc4e-a72fe1377a5a@jansson.tech/
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-08-16 15:18:11 +02:00
David Sterba 9f2bfd966e btrfs-progs: convert: rename context volume_name to label
The name was derived from ext2 but we use label elsewhere.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-08-20 14:24:55 +02:00
David Sterba bcae45d9e6 btrfs-progs: convert: new option to copy or specify uuid
Add new option --uuid to convert with the following modes:

- 'copy' -- copy the UUID from the source filesystem
- 'new' -- (default) generate new UUID
- UUID -- a valid UUID that will be set on btrfs

Based on patch from Florian

https://lore.kernel.org/linux-btrfs/1357486331-4615-2-git-send-email-falbrechtskirchinger@gmail.com/

and ported to contemporary codebase.

Issue: #391
Signed-off-by: David Sterba <dsterba@suse.com>
2021-08-20 14:24:55 +02:00
David Sterba b1f374dd1d btrfs-progs: switch %Lu to %llu format
The %Lu format is not standard and we use %llu everywhere else, so
switch the remaining cases.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-06-19 22:07:49 +02:00
Pierre Labastie 1b49370973 btrfs-progs: build: fix the test for EXT4_EPOCH_MASK
Commit b3df561fbf ("btrfs-progs: convert: copy extra timespec on
ext4") has introduced the ability to convert extended inode time
precision on ext4, but this breaks builds on older distros, where ext4
does not have the nsec time precision.

Commit c615287cc0 ("btrfs-progs: a bunch of typo fixes") tried to fix
that by testing the availability of the EXT4_EPOCH_MASK macro, but the
test is not complete.

This patch aims at fixing the macro test, and changes the
name of the associated HAVE_ macro, since the logic is reverted.

This fixes #353 when ext4 has nsec time precision. Note that the test
convert/019-ext4-copy-timestamps fails when ext4 does not have the nsec
time precision and needs to check for the support.

Issue: #353
Signed-off-by: Pierre Labastie <pierre.labastie@neuf.fr>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-03-23 17:04:22 +01:00
David Sterba 2c53231af9 btrfs-progs: convert: refuse to convert filesystem with 'needs_recovery' set
As Chris reports: This ext4 file system has 'needs_recovery' feature set, and
if mounted rw, log replay happens. But btrfs-convert doesn't check for it and
converts anyway. It probably shouldn't.

  # debugfs -R stats /dev/loop0
  debugfs 1.45.6 (20-Mar-2020)
  Filesystem volume name:   <none>
  Last mounted on:          /mnt/0
  Filesystem UUID:          d3e3862e-f892-4ab7-ae91-84eb4be4a3ef
  Filesystem magic number:  0xEF53
  Filesystem revision #:    1 (dynamic)
  Filesystem features:      has_journal ext_attr resize_inode dir_index
			    filetype needs_recovery extent 64bit flex_bg
			    sparse_super large_file huge_file dir_nlink
			    extra_isize metadata_csum
  Filesystem flags:         signed_directory_hash
  Default mount options:    user_xattr acl
  Filesystem state:         clean
  Errors behavior:          Continue
  ...

Then 'btrfs-convert' proceeds, while 'e2fsck -fvn /dev/loop1' finds some
problems and wants to fix them.

Add a check for the 'needs_recovery' incompat bit set and don't convert
the filesystem.

Issue: #348
Signed-off-by: David Sterba <dsterba@suse.com>
2021-03-15 15:40:54 +01:00
David Sterba 405e36d53c btrfs-progs: convert: check for extra timespec support in e2fsprogs
In 5.10 the convert gained support for extended inode time precision,
but this is not available on older distros and breaks build. Add a
configure-time check for the EXT4_EPOCH_MASK macro and add a stub in
case it's not detected.

This means that the 64bit timestamps will not be transferred from the
original filesystem in such environment, at least a warning is printed.

Issue: #344
Signed-off-by: David Sterba <dsterba@suse.com>
2021-02-24 16:40:11 +01:00
Jiachen YANG b3df561fbf btrfs-progs: convert: copy extra timespec on ext4
Currently btrfs-convert only copies ext2 inode timestamps
i_[cma]time from ext4, while filling 0 to nsec and crtime fields.

This change copies nsec and crtime by parsing i_[cma]time_extra fields.

Author: Jiachen YANG <farseerfc@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-01-13 22:33:10 +01:00
David Sterba 6069bc52a9 btrfs-progs: move transaction.c to kernel-shared/
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:06 +02:00
David Sterba 772f0da6df btrfs-progs: move disk-io.c to kernel-shared/
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:05 +02:00
Qu Wenruo b79fdc0ac3 btrfs-progs: convert: handle errors better in ext2_copy_inodes()
This patch will enhance the error handling of ext2_copy_inodes by:

- Return more meaningful error number
  Instead of -1 (-EPERM), now return -EIO for ext2 calls error, and
  proper error number from btrfs calls.

- Commit transaction if ext2fs_open_inode_scan() failed

- Call ext2fs_close_inode_scan() on error

- Hunt down the BUG_ON()s

- Add error messages for transaction related calls

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:03 +02:00
Qu Wenruo c9c4eb1f3f btrfs-progs: convert: prevent 32bit overflow for cctx->total_bytes
[BUG]
When convert is called on a 64GiB ext4 fs, it fails like this:

  $ btrfs-convert  /dev/loop0p1
  create btrfs filesystem:
          blocksize: 4096
          nodesize:  16384
          features:  extref, skinny-metadata (default)
          checksum:  crc32c
  creating ext2 image file
  ERROR: missing data block for bytenr 1048576
  ERROR: failed to create ext2_saved/image: -2
  WARNING: an error occurred during conversion, filesystem is partially created but not finalized and not mountable

Btrfs-convert also corrupts the source fs:

  $ LANG=C e2fsck /dev/loop0p1 -f
  e2fsck 1.45.6 (20-Mar-2020)
  Resize inode not valid.  Recreate<y>? yes
  Pass 1: Checking inodes, blocks, and sizes
  Deleted inode 3681 has zero dtime.  Fix<y>? yes
  Inodes that were part of a corrupted orphan linked list found.  Fix<y>? yes
  Inode 3744 was part of the orphaned inode list.  FIXED.
  Deleted inode 3745 has zero dtime.  Fix<y>? yes
  Inode 3747 has INLINE_DATA_FL flag on filesystem without inline data support.
  Clear<y>? yes
  ...

[CAUSE]
After some debugging, the first strange behavior is, the value of
cctx->total_bytes is 0 in ext2_open_fs().

It turns out that, the value assign for cctx->total_bytes could lead to
bit overflow for the unsigned int value.

And that 0 cctx->total_bytes leads to various problems for later free
space calculation.
For example, in calculate_available_space(), we use cctx->total_bytes to
ensure we won't create a data chunk beyond device end:

		cue_len = min(cctx->total_bytes - cur_off, cur_len);

If that cur_offset is also 0, we will create a cache_extent with 0 size,
which could cause a lot of problems for cache tree search.

[FIX]
Do manual casting for the multiply operation, so we could got a real u64
result.  The fix will be applied to all supported fses (ext* and
reiserfs).

Reported-by: Christian Zangl <coralllama@gmail.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-28 18:31:06 +02:00
Qu Wenruo ac75513621 btrfs-progs: convert: fix the pointer sign warning for ext2 label
[WARNING]
When compiling btrfs-progs, there is one warning from convert ext2 code:
  convert/source-ext2.c: In function 'ext2_open_fs':
  convert/source-ext2.c:91:44: warning: pointer targets in passing argument 1 of 'strndup' differ in signedness [-Wpointer-sign]
     91 |  cctx->volume_name = strndup(ext2_fs->super->s_volume_name, 16);
        |                              ~~~~~~~~~~~~~~^~~~~~~~~~~~~~~
        |                                            |
        |                                            __u8 * {aka unsigned char *}
  In file included from ./kerncompat.h:25,
                   from convert/source-ext2.c:19:
  /usr/include/string.h:175:35: note: expected 'const char *' but argument is of type '__u8 *' {aka 'unsigned char *'}
    175 | extern char *strndup (const char *__string, size_t __n)
        |                       ~~~~~~~~~~~~^~~~~~~~

The toolchain involved is:
- GCC 10.1.0
- e2fsprogs 1.45.6

[CAUSE]
Obviously, in the offending e2fsprogs, the volume label is using u8,
which is unsigned char, not char.

  /*078*/	__u8	s_volume_name[EXT2_LABEL_LEN];	/* volume name, no NUL? */

[FIX]
Just do a forced conversion to suppress the warning is enough.
I don't think we need to apply -Wnopointer-sign yet.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-06-29 17:45:39 +02:00
David Sterba c07960c8be btrfs-progs: move utils.[ch] to common/
Update include paths and remove some duplicates.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:04 +02:00
Qu Wenruo 50e3858869 btrfs-progs: convert: Workaround delayed ref bug by limiting the size of a transaction
In convert we use trans->block_reserved >= 4096 as a threshold to commit
transaction, where block_reserved is the number of new tree blocks
allocated inside a transaction.

The problem is, we still have a hidden bug in delayed ref implementation
in btrfs-progs, when we have a large enough transaction, delayed ref may
failed to find certain tree blocks in extent tree and cause transaction
abort.

This fix will workaround it by committing transaction at a much lower
threshold.

The old 4096 means 4096 new tree blocks, when using default (16K)
nodesize, it's 64M, which can contain over 12k inlined data extent or
csum for around 60G, or over 800K file extents.

The new threshold will limit the size of new tree blocks to 2M, aligning
with the chunk preallocator threshold, and reducing the possibility to
hit that delayed ref bug.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-05-27 16:04:02 +02:00
David Sterba e578b59bf6 btrfs-progs: convert strerror to implicit %m
Similar to the changes where strerror(errno) was converted, continue
with the remaining cases where the argument was stored in another
variable.

The savings in object size are about 4500 bytes:

 $ size btrfs.old btrfs.new
   text    data     bss     dec     hex filename
 805055   24248   19748  849051   cf49b btrfs.old
 804527   24248   19748  848523   cf28b btrfs.new

Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-31 18:24:14 +01:00
Qu Wenruo e02049d964 btrfs-progs: convert: Fix inline file extent creation condition
[Bug]
On btrfs converted from ext*, one user reported the following kernel
warning:
 ------------[ cut here ]------------
 BTRFS: Transaction aborted (error -95)
 WARNING: CPU: 0 PID: 324 at fs/btrfs/inode.c:3042 btrfs_finish_ordered_io+0x7ab/0x850 [btrfs]
 Workqueue: btrfs-endio-write btrfs_endio_write_helper [btrfs]
 RIP: 0010:btrfs_finish_ordered_io+0x7ab/0x850 [btrfs]
 ...
 Call Trace:
  normal_work_helper+0x39/0x370 [btrfs]
  process_one_work+0x1ce/0x410
  worker_thread+0x2b/0x3d0
  ? process_one_work+0x410/0x410
  kthread+0x113/0x130
  ? kthread_create_on_node+0x70/0x70
  ? do_syscall_64+0x74/0x190
  ? SyS_exit_group+0x10/0x10
  ret_from_fork+0x35/0x40
 ---[ end trace c8ed62ff6a525901 ]---
 BTRFS: error (device dm-2) in
btrfs_finish_ordered_io:3042: errno=-95 unknown
 BTRFS info (device dm-2): forced readonly
 BTRFS error (device dm-2): pending csums is 6447104

[Cause]
The call trace and the unique return value points to
__btrfs_drop_extents(), when we tries to drop pages of an inline extent,
we will trigger such -EOPNOTSUPP.

However kernel has limitation on the size of inline file extent
(sector size for ram size and sector size - 1 for on-disk size),
btrfs-convert doesn't have the same limitation, resulting much larger
file extent.

The lack of correct inline extent size check dates back to 2008 when
btrfs-convert is added into btrfs-progs.

[Fix]
Fix the inline extent creation condition, not only using
BTRFS_MAX_INLINE_DATA_SIZE(), which is only the maximum size of inline
data according to nodesize, but also limit it against sector size.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-03-30 22:15:54 +02:00
Qu Wenruo 92ffad47b5 btrfs-progs: convert/ext2: Remove check for ext2_ext_attr_entry->e_value_block
In latest e2fsprogs (1.44.0) definition of ext2_ext_attr_entry has
removed member e_value_block, as currently ext* doesn't support it set
anyway.

So remove such check so that we can pass compile.

Issue: #110
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=199071
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-03-30 22:15:54 +02:00
Qu Wenruo b8e884f1cd btrfs-progs: convert/ext2: Fix memory leak caused by handled ext2_filsys
Exposed by convert-test with D=asan.

Unlike btrfs, ext2fs_close() still leaves its ext2_filsys parameter
filled with allocated pointers.

It needs ext2fs_free() to free those pointers.

Issue: #92
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-02-14 16:33:31 +01:00
Gu Jinxiang c958110785 btrfs-progs: Sync code with kernel for BTRFS_MAX_INLINE_DATA_SIZE
Do a cleanup. Also make it consistent with kernel.  Use fs_info instead
of root for BTRFS_MAX_INLINE_DATA_SIZE, since maybe in some situation we
do not know root, but just know fs_info.

Change macro to inline function to be consistent with kernel.  And
change the function body to match kernel.

Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-02-02 16:01:57 +01:00
Gu Jinxiang 26072f584d btrfs-progs: Use fs_info instead of root for BTRFS_LEAF_DATA_SIZE
Do a cleanup. Also make it consistent with kernel.  Use fs_info instead
of root for BTRFS_LEAF_DATA_SIZE, since maybe in some situation we do
not know root, but just know fs_info.

Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-02-02 16:01:56 +01:00
David Sterba 29faad6339 btrfs-progs: convert: use correct string printing for errcode_t
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:17:21 +02:00
David Sterba 87adfe8254 btrfs-progs: convert: don't print message when fs is not recognized, ext2
Bad magic error means it's not the desired filesystem so the error
message is just noise.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba 448999d84d btrfs-progs: add crude error handling when transaction start fails
Currently transaction bugs out insided btrfs_start_transaction in case
of error, we want to lift the error handling to the callers. This patch
adds the BUG_ON anywhere it's been missing so far. This is not the best
way of course. Transforming BUG_ON to a proper error handling highly
depends on the caller and should be dealt with case by case.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba 4856215f51 btrfs-progs: convert: move and rename dev_t helpers to common file
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba c05c70c77c btrfs-progs: convert: move acl helper to common source file
There were 2 copies of ext2_acl_count and acl_ea_size.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
Adam Buchbinder 1639cc2a51 btrfs-progs: convert: Fix data race when reporting progress
The status display was reading the state while the task was updating
it. Use a mutex to prevent the race.

This race was detected using ThreadSanitizer and
misc-tests/005-convert-progress-thread-crash.

==================
WARNING: ThreadSanitizer: data race
  Write of size 8 by main thread:
    #0 ext2_copy_inodes btrfs-progs/convert/source-ext2.c:853
    #1 copy_inodes btrfs-progs/convert/main.c:145
    #2 do_convert btrfs-progs/convert/main.c:1297
    #3 main btrfs-progs/convert/main.c:1924

  Previous read of size 8 by thread T1:
    #0 print_copied_inodes btrfs-progs/convert/main.c:124

  Location is stack of main thread.

  Thread T1 (running) created by main thread at:
    #0 pthread_create <null>
    #1 task_start btrfs-progs/task-utils.c:50
    #2 do_convert btrfs-progs/convert/main.c:1295
    #3 main btrfs-progs/convert/main.c:1924

SUMMARY: ThreadSanitizer: data race
btrfs-progs/convert/source-ext2.c:853 in ext2_copy_inodes

Signed-off-by: Adam Buchbinder <abuchbinder@google.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-07-20 17:43:43 +02:00
Qu Wenruo ec5a427fec btrfs-progs: Refactor sectorsize users in convert/source-ext2.c
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
2017-07-03 13:35:11 +02:00
Liu Bo 188f9c889a Btrfs-progs: convert: do not clear nodatasum flag in inode item
With the current btrfs-convert, if we convert a ext4 without data checksum,
it'd not set nodatasum flag in inode item, nor create csum item, reading
file ends up with checksum errors.

Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-06-30 16:31:02 +02:00
Lakshmipathi.G 324d4c1857 btrfs-progs: convert: Add larger device support
With larger file system (in this case its 22TB), ext2fs_open() returns
EXT2_ET_CANT_USE_LEGACY_BITMAPS error message with
ext2fs_read_block_bitmap().

To overcome this issue,
(a) we need pass EXT2_FLAG_64BITS flag with ext2fs_open.
(b) use 64-bit functions like ext2fs_get_block_bitmap_range2,
    ext2fs_inode_data_blocks2,ext2fs_read_ext_attr2
(c) use 64bit types with btrfs_convert_context fields

Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=194795
Signed-off-by: Lakshmipathi.G <lakshmipathi.g@giis.co.in>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-06-26 17:09:54 +02:00
Baruch Siach 153d4e6c01 btrfs-progs: add missing limits headers
Build under musl libc fails because of missing PATH_MAX and XATTR_NAME_MAX
macro declarations. Add the required headers.

Signed-off-by: Baruch Siach <baruch@tkos.co.il>
2017-03-16 17:02:44 +01:00
David Sterba 52319450e7 btrfs-progs: convert: rename members that clash with other functions
Rename 'free' and also 'used' for consistency.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-03-08 13:00:46 +01:00
David Sterba 39aa42147a btrfs-progs: convert: rename ext2 function to create a symlink
Signed-off-by: David Sterba <dsterba@suse.com>
2017-03-08 13:00:46 +01:00
David Sterba 0c362b9031 btrfs-progs: convert: use bit field for convert flags
Use one flag field instead of several variables. The change cascades
down to the callchain and modifies several functions.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-03-08 13:00:46 +01:00
David Sterba c45a060f3e btrfs-progs: convert: use helper for special inode number check
Signed-off-by: David Sterba <dsterba@suse.com>
2017-03-08 13:00:46 +01:00
David Sterba 9368a77456 btrfs-progs: convert: better error handling in ext2_read_used_space
Signed-off-by: David Sterba <dsterba@suse.com>
2017-03-08 13:00:46 +01:00
David Sterba 03c085861b btrfs-progs: convert: move ext2 conversion out of main.c
Signed-off-by: David Sterba <dsterba@suse.com>
2017-03-08 13:00:45 +01:00