Add helper scripts to build static binaries manually or clean old runs.
Requires the 'gh' command line tool and write access the git repository.
Signed-off-by: David Sterba <dsterba@suse.com>
Add new CI image target based on 32bit Alpine with musl. It's a copy of
musl and requires docker's buildx extension to properly select the
platform.
Signed-off-by: David Sterba <dsterba@suse.com>
Calculate checksums for static binaries that are published as release
artifacts. Do that when they're built in the workflow so the checksum
appears in the build log and also provide the artifacts of the checksums
so they can be published along the releases.
Signed-off-by: David Sterba <dsterba@suse.com>
Rename the options so they more accurately reflect what the command is
actually doing. The feature is enabled/disabled in the end but it's not
a simple on/off like for others, the conversion takes time.
Signed-off-by: David Sterba <dsterba@suse.com>
There has been no update since 2014 and it's likely broken due to many
new features added to btrfs since then.
Signed-off-by: David Sterba <dsterba@suse.com>
- "Wipe" in storage terms is often understood as some kind of secure
deletion. Use "remove" instead in order to indicate that the space
cache is fully removed (and not just cleared and then e.g.
automatically rebuild).
- The --clear-space-cache option for btrfs check actually clears the
whole space cache, just as documented. Thus move any documentation
about the clear_cache mount option not doing so for v1 to that.
Instead, refer to the mount option.
- Also note that when clear_cache is used with v1, the free space cache
for block groups that are modified gets always cleared, but rebuilt
only if nospace_cache is not used.
Pull-request: #571
Signed-off-by: Christoph Anton Mitterer <mail@christoph.anton.mitterer.name>
Signed-off-by: David Sterba <dsterba@suse.com>
Clarify active vs. inactive swapfile regarding snapshotting and further minor
improvements.
Issue: #608
Pull-request: #618
Signed-off-by: Christoph Anton Mitterer <mail@christoph.anton.mitterer.name>
Signed-off-by: David Sterba <dsterba@suse.com>
Currently "btrfs filesystem du" auto-completes for directories only,
but it can also be used against files to determine shared vs exclusive
extents.
Signed-off-by: David Disseldorp <ddiss@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
The combination of --force and --repair was mistakenly documented as not
working on a mounted filesystem. Update help text, documentation and the
warning itself.
Issue: #503
Signed-off-by: David Sterba <dsterba@suse.com>
Running 'btrfs inspect tree-stats' on a mounted filesystem works though
it reads directly from block devices. This can lead to inconsistent
data, warnings, errors or or a crash. More checks could be added but at
least explain things in more detail.
Issue: #520
Signed-off-by: David Sterba <dsterba@suse.com>
Attempting to create a snapshot of subvolume with an active swapfile
prints the errno message corresponding to ETXTBSY but this is confusing
so change it to be more descriptove to:
ERROR: cannot snapshot '/hibernate': source subvolume contains an active swapfile (Text file busy)
Issue: #607
Signed-off-by: David Sterba <dsterba@suse.com>
The *64 interfaces, such as fstat64, off64_t, etc, are legacy interfaces
created at a time when 64-bit file support was still new. They are
generally exposed when defining a macro named _LARGEFILE64_SOURCE, as
e.g. the glibc docs[0] say.
The modern way to utilise largefile support, is to continue to use the
regular interfaces (off_t, fstat, ..), and define _FILE_OFFSET_BITS=64.
We already use the autoconf macro AC_SYS_LARGEFILE[1] which arranges this
and sets this macro for us. Therefore, we can utilise the non-64 names
without fear of breaking on 32-bit systems.
This fixes the build against musl libc, ever since musl dropped the
*64 compat from interfaces by default[2] just for _GNU_SOURCE, unless
_LARGEFILE64_SOURCE is defined. However, there are plans for a future
removal of the whole *64 header API, and that workaround (adding another
define) might cease to exist.
So, rename all *64 API use to the regular non-suffixed names. For
consistency, rename the internal functions that were *64 named
(lstat64_path, ..) too.
This should have no regressions on any platform.
[0]: https://www.gnu.org/software/libc/manual/html_node/Feature-Test-Macros.html#index-_005fLARGEFILE64_005fSOURCE
[1]: https://www.gnu.org/software/autoconf/manual/autoconf-2.67/html_node/System-Services.html
[2]: 25e6fee27f
Pull-request: #615
Signed-off-by: psykose <alice@ayaya.dev>
Signed-off-by: David Sterba <dsterba@suse.com>
Executing the script inside the directories as './test.sh' is not
supposed to work but could happen accidentally. With an exit after
attempting to source the we can fix that.
Signed-off-by: David Sterba <dsterba@suse.com>
We've changed mkfs defaults in 5.15 so it's not necessary to test the
features separately with convert. Instead use only defaults and add
other features that can be selected independently.
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
There is an internal bug report that, after mkfs.btrfs there is a chance
that no /dev/disk/by-uuid/<uuid> symlink is not created at all.
[CAUSE]
That uuid symlink is created by udev, which listens to inotify
IN_CLOSE_WRITE events from all block devices.
After such IN_CLOSE_WRITE event is triggered, udev would *disable*
inotify for that block device, and do a blkid scan on it.
After the blkid scan is done, re-enables the inotify listening.
This means normally mkfs tools should open the fd, do all the writes,
and close the fd after everything is done.
But unfortunately for mkfs.btrfs, it's not the case, we have a lot of
phases separated by different close() calls:
open_ctree() would open fds of each involved device
and close them at close_ctree()
Only after close_ctree() we have a valid superblock -\
|
|<------- A -------->|<--------- B --------->|<------- C ------->|
| |
| `- open a new fd for make_btrfs()
| and close it before open_ctree()
| The device contains invalid sb.
|
`- open a new fd for each device, then call
btrfs_prepare_device(), then close the fd.
The device would contain no valid superblock.
If at the close() of phase A udev event is triggered, while doing udev
scan we go into phase C (but before the new valid super blocks written),
udev would only see no superblock or invalid superblock.
Then phase C finished, udev resumes its inotify listening, but at this
time mkfs is finished, while udev only sees the premature data from
phase A, and misses the IN_CLOSE_WRITE events from phase C.
[FIX]
Instead of opening and closing a new fd for each device, re-use the fd
opened during prepare_one_device(), and close all the fds until
close_ctree() is called.
By this, although we may still have race between close_ctree() and
explicit close() calls, at least udev can always see the properly
written super blocks.
To compensate the change, some extra cleanups are made:
- Do not touch @device_count
Which makes later prepare_ctx iteration much easier.
- Remove top-level @fd variable
Instead go with prepare_ctx[i].fd.
- Do not open with O_RDWR in test_dev_for_mkfs()
as test_dev_for_mkfs() would close the fd, if we go O_RDWR, it can
cause the udev race.
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
With previous btrfstune support to convert to block-group-tree, it has
implemented most of the infrastructure for bi-directional conversion.
This patch will implement the remaining conversion support to go back to
extent tree.
The modification includes:
- New convert_to_extent_tree() function in btrfstune.c
It's almost the same as convert_to_bg_tree(), but with small changes:
* No need to set extra features like NO_HOLES/FST.
* Need to delete the block group tree when everything finished.
- Update btrfs_delete_and_free_root() to handle non-global roots
Currently the function can only accepts global roots (extent/csum/free
space trees)
If we pass a non-global root into the function, we will screw up
global_roots_tree and crash.
Since we're going to use btrfs_delete_and_free_root() to free block
group tree which is not a global tree, this is needed.
- New handling for half converted fs in get_last_converted_bg()
There are two cases need to be handled:
* The bg tree is already empty
We need to grab the first bg in extent tree.
Or at conversion function we will fail at grabbing the first bg.
* The bg tree is not empty
Then we need to grab the last bg in extent tree.
- Extra root switching in involved functions. This involves:
* read_converting_block_groups()
* insert_block_group_item()
* update_block_group_item()
We just need to update our target root according to the current
compat_ro and super flags.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The free tier of travis-ci has ended on 05/2021, remove the scripts.
Nowadays zstd has made it to all distros so we don't need to build
it from sources, reiserfs support is fading out so we don't need to
build it either. The integration with coverity scan needs to be done
again. The rest has been moved to new github actions CI.
Signed-off-by: David Sterba <dsterba@suse.com>
The gitlab CI was experimentally used for btrfs-progs testsuite in
nested VM with recent kernel (as the base CI image was too old). This
unfortunately did not work due to time limit for the CI run, the build
and tests haven't finished.
Gitlab supports self-hosted runners so this can be integrated
eventually.
Signed-off-by: David Sterba <dsterba@suse.com>
The block-group-tree used to be under experimental flag in -R but now
that we've deprecated -R it does not make sense to leave
block-group-tree there for compatibility, this has never been exposed to
users.
Signed-off-by: David Sterba <dsterba@suse.com>
The feedback from the community on block group tree is very positive,
the only complain is, end users need to recompile btrfs-progs with
experimental features to enjoy the new feature.
So let's move it out of experimental features and let more people enjoy
faster mount speed.
Also change the option of btrfstune, from `-b` to
`--enable-block-group-tree` to avoid short option.
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The option -R|--runtime-features was introduced to support features that
don't result in a full incompat flag change, thus things like
free-space-tree and quota features are put here.
But to end users, such separation of features is not helpful and can be
sometimes confusing.
Thus we're already migrating those runtime features into -O|--features
option under experimental builds.
I believe this is the proper time to move those runtime features into
-O|--features option, and mark the -R|--runtime-features option
deprecated.
For now we still keep the old option as for compatibility purposes.
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Build static binaries (published as artifacts):
- btrfs.static - the main tool
- btrfs.box.static - all-in-one tool (functionality selected by file name)
There are a few problems:
- unstable URL, i.e. not one location to get the binaries from, multiple
steps need on web or by using API
- downloaded files are in .zip (compression cannot be disabled with
upload-artifact@v3)
- permissions of the file inside .zip are lost, manual chmod needed
after extraction
- artifacts can be downloaded only for logged users
Issue: #593
Signed-off-by: David Sterba <dsterba@suse.com>
For a release test check that all compatibility build targets work. This
may take long as it's pulling lots of data (no caching yet) so it's not
run on devel.
Signed-off-by: David Sterba <dsterba@suse.com>
Using 'docker run -it' works for interactive sessions but not inside the
CI environment. We want both so make it optional by detecting if stdin
is a terminal.
Signed-off-by: David Sterba <dsterba@suse.com>
There's an unconditional 'shift' in the scripts that will lead to an
error "cannot shift" and this would fail inside CI. We want an implicit
run on current HEAD so shift only when there are more parameters.
Signed-off-by: David Sterba <dsterba@suse.com>
There are some helpers that invoke setup_root_helper internally so it's
not needed to run test.sh without root, but it should be there in case
the test calls SUDO_HELPER so it's always paired.
Signed-off-by: David Sterba <dsterba@suse.com>
We need to call the setup_root_helper before we start messing with the
loop devices.
Reviewed-by: Neal Gompa <neal@gompa.dev>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We missed a couple of $SUDO_HELPER uses in this test that made it
impossible to run without root. Add them in so we can run as a normal
user.
Reviewed-by: Neal Gompa <neal@gompa.dev>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The setup_root_helper needs to be called before messing with the loop
devices, and btrfs check needs to be run with $SUDO_HELPER.
Reviewed-by: Neal Gompa <neal@gompa.dev>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We need to make sure the root helper is setup before calling the loop
helpers, and additionally we need to use $SUDO_HELPER when we run the
final btrfs check. With this patch we can now run this test as a normal
user.
Reviewed-by: Neal Gompa <neal@gompa.dev>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
- convert "*" lists to "-" for newer releases, this is preferred in pull
requests
- update nested lists to the more compact layout (add more newlines,
un-indent)
- wrap overly long lines
Signed-off-by: David Sterba <dsterba@suse.com>
