Commit graph

303 commits

Author SHA1 Message Date
David Sterba 82464a03e7 btrfs-progs: add more hash implementation providers
For environments that require certified implementations of cryptographic
primitives allow to select a library providing them. The requirements
are SHA256 and BLAKE2 (with the 2b variant and 256 bit digest).

For now there are two: libgrcrypt and libsodium (openssl does not
provide the BLAKE2b-256). Accellerated versions are typically provided
and automatically selected.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-05-04 20:48:40 +02:00
David Sterba 8c85b34420 btrfs-progs: move backref.[ch] to kernel-shared/
The files are very close to kernel versions, for that keep them in the
shared directory.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-05-04 20:48:35 +02:00
David Sterba 7b2712bf92 btrfs-progs: build: add target for check lowmem mode
Running the lowmem mode for check needs some setup and is not usually
tested, so add a new target that sets up the variables.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:36 +02:00
David Sterba 58bcd4260f btrfs-progs: move free-space-tree.[ch] to kernel-shared/
The files are very close to kernel versions, for that keep them in the
shared directory.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:34 +02:00
David Sterba 019489a143 btrfs-progs: move delayed-ref.[ch] to kernel-shared/
The files are very close to kernel versions, for that keep them in the
shared directory.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:34 +02:00
David Sterba 49fe198218 btrfs-progs: move ioctl-test.c to tests/
Move it out of the top-level directory.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:34 +02:00
David Sterba b147ef8764 btrfs-progs: move library-test.c to tests/
Move it out of the top-level directory.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:34 +02:00
David Sterba 7c2c1dfcfc libbtrfsutil: add list of exported symbols for version 1.1.1
The library exports should have the symbols versioned to allow future
extensions. The earlier versions are skipped and current version is the
first one, no change for existing applications as the same set of
functions is exported.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-04 19:21:09 +01:00
Marcos Paulo de Souza 225d66e99b btrfs-progs: Include btrfs-find-root and btrfs-select-super in testsuite
These two binaries are not usually shipped with the package, but they
are used by the testsuite, they need to be included in the exported tar.

Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-03 19:58:54 +01:00
David Sterba c56439735a btrfs-progs: build: document install targets
Enumerate what exactly gets installed.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-22 19:09:52 +01:00
Fabrice Fontaine 9a03f6d41f btrfs-progs: build: install static library and headers in install-static
Currently, the install-static target only installs the statically
compiled tools. However, some programs outside of btrfs-progs (for
example docker) link with the btrfs libraries. If such programs want to
link statically against the btrfs-progs library, then this library
should be installed by "install-static". Indeed, "make install" cannot
be used if the support for shared library is not enabled.

Pull-request: #211
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
[Gustavo: Rebase for btrfs-progs 4.7.2]
Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
[Vincent: Rebase for btrfs-progs 4.10]
Signed-off-by: Vicente Olivert Riera <Vincent.Riera@imgtec.com>
[baruch: Rebase for btrfs-progs 4.16]
Signed-off-by: Baruch Siach <baruch@tkos.co.il>
[Retrieved from:
https://git.buildroot.net/buildroot/tree/package/btrfs-progs/0001-Makefile-install-static-library-and-headers-in-insta.patch]
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-22 19:09:52 +01:00
Marcos Paulo de Souza c052b38418 btrfs-progs: Makefile: Add -Wimplicit-fallthrough
Avoid introducing new cases of implicit fallthrough by having this flag
always set, though a conditional check is needed to avoid build breakage
on older compilers or on CI.

Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:21:08 +01:00
David Sterba 3778ece7ff btrfs-progs: add blake2b reference implementation
Upstream commit 997fa5ba1e14b52c554fb03ce39e579e6f27b90c,
git repository: git://github.com/BLAKE2/BLAKE2

The reference implemetation added in this patch is unchanged and will be
modified only to compile in current code base and with minimal other
modifications in case of future sync with upstream code. IOW, the coding
style should stay as-is and does not conform to the other btrfs-progs
code. This is an exception for xxhash and sha256 code as well.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:21:05 +01:00
Johannes Thumshirn ae9f8bff30 btrfs-progs: add sha256 as supported checksumming algorithm
Add the definition to the checksum types and let mkfs accept it.

Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:20:40 +01:00
David Sterba 785073f658 btrfs-progs: crypto: add hash speedtest utility
A simple tool to microbenchmark performance of the hashes. Uses rdtsc
for timing, so works only on x86_64.

 $ make hash-speedtest
 $ ./hash-speedtest [iterations]

   Block size: 4096
   Iterations: 100000

       NULL-NOP: cycles:     56061823, c/i      560
    NULL-MEMCPY: cycles:     61296469, c/i      612
	 CRC32C: cycles:    179961796, c/i     1799
	 XXHASH: cycles:    138434590, c/i     1384

Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:20:03 +01:00
David Sterba 3a73bc1b37 btrfs-progs: move sha256 from tests to crypto/
The SHA256 is going to be used in the future, so this makes it a second
user and we also have the appropriate directory now.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:20:02 +01:00
David Sterba bc2fff4675 btrfs-progs: build: clean temporary files in crypto/
The directory crypto/ is new, so we need to add it to the clean targets.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:20:02 +01:00
Johannes Thumshirn c04bcdcacc btrfs-progs: move crc32c implementation to crypto/
With the introduction of xxhash64 to btrfs-progs we created a crypto/
directory for all the hashes used in btrfs (although no
cryptographically secure hash is there yet).

Move the crc32c implementation from kernel-lib/ to crypto/ as well so we
have all hashes consolidated.

Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:20:02 +01:00
Johannes Thumshirn f070ece2e9 btrfs-progs: add xxhash64 to mkfs
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:20:00 +01:00
David Sterba 075f147cef btrfs-progs: preload libbtrfs for libbtrfs-test
The libbtrfs-test simulated build happens outside of the source
repository, but sometimes the system library is used instead of the repo
one. When -rpath does not work, force the correct library by LD_PRELOAD.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-10-25 13:03:54 +02:00
David Sterba c15c559f54 btrfs-progs: build: add missing symbols to libbtrfs
Several people reported build breakage of snapper, due to missing
symbols in libbtrfs.so. Move the objects to the library objects, now we
don't have to worry about the new exports as the libbtrfs.sym is
unchanged. And there are no new .h files being exported though there are
the .o files in the library.

Issue: #214
Link: https://github.com/openSUSE/snapper/issues/500
Signed-off-by: David Sterba <dsterba@suse.com>
2019-10-25 13:03:54 +02:00
David Sterba c618f46222 btrfs-progs: libbtrfs: add list of exported symbols
The shared library exports many functions that are not supposed to be
public, like rb-tree, crc32c or internal helpers but as this has been
potentially in use we should at least make a list.  There's only a
subset being used by the snapper project.

Export majority of current symbols visible in libbtrfs so any future
additions to libbtrfs objects are automatically hidden and don't pollute
the namespace further.

Note that all projects should switch to libbtrfsutil rather than
libbtrfs that exists for historical reasons and will be deprecated in
the future.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-10-24 16:31:25 +02:00
David Sterba e2da195a3c btrfs-progs: build: add missing objects to libbtrfs
A user reports that some symbols are missing from libbtrfs, eg.
radix_tree_init. This is correct and there are few more. The headers
exported through the library all need the respective object files.

The sources are GPL so is libbtrfs, which is known

Issue: #205
Signed-off-by: David Sterba <dsterba@suse.com>
2019-10-14 17:30:56 +02:00
David Sterba 8e68287d88 btrfs-progs: move qgroup-verify.[ch] to check/
The only user of the functionality is in check.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-10-14 17:27:35 +02:00
David Sterba bd4a386ec5 btrfs-progs: build most common tools into one binary (busybox style)
Build several standalone tools into one binary and switch the function
by name (symlink or hardlink).

* btrfs
* mkfs.btrfs
* btrfs-image
* btrfs-convert
* btrfstune

The static target is also supported. The name of resulting boxed
binaries is btrfs.box and btrfs.box.static . All the binaries can be
built at the same time without prior configuration.

   text    data     bss     dec     hex filename
 822454   27000   19724  869178   d433a btrfs
 927314   28816   20812  976942   ee82e btrfs.box
2067745   58004   44736 2170485  211e75 btrfs.static
2627198   61724   83800 2772722  2a4ef2 btrfs.box.static

File sizes:

  857496  btrfs
  968536  btrfs.box
 2141400  btrfs.static
 2704472  btrfs.box.static

Standalone utilities:

  512504  btrfs-convert
  495960  btrfs-image
  471224  btrfstune
  491864  mkfs.btrfs

 1747720  btrfs-convert.static
 1411416  btrfs-image.static
 1304256  btrfstune.static
 1361696  mkfs.btrfs.static

So the shared 900K binary saves ~2M, or ~5.7M for static build.

Signed-off-by: David Sterba <dsterba@suse.cz>
2019-07-04 15:30:40 +02:00
David Sterba a62b7854de btrfs-progs: utils: split device scanning functions to own file
All helpers and data structures that are used for device registration.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-04 02:06:34 +02:00
David Sterba ccbea0977b btrfs-progs: utils: split device handling functions to own file
Helpers that read size, do zeoring, trim or prepare/finalize the device.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-04 02:06:34 +02:00
David Sterba f83e81c61b btrfs-progs: utils: split path related utils to own file
Group helpers that return some status of a given path to own file so we
debloat utils.c a bit.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-04 02:06:31 +02:00
David Sterba 3122085a79 btrfs-progs: output formatter infrastructure
Add structures and API for unified output definition and multiple
formatting backends. Currently there's plain text and json.

The format of each row is defined in struct rowspec, selected using a
key and formatted according to the type. There are extended types for
eg. UUID or pretty size, while direct printf format specifiers work too.

Due to different nature of the outputs, the context structure members
are not always used.

* text output mostly uses indentation and formats the name to a given
  width

* json output tracks nesting depth and keeps stack of previous groups
  (list or array) and how many member have been printed, as the
  separators are allowed only between values and must not preced the
  group closing bracket
  the nesting depth is hardcoded to 16, counting the global group

The API provides functions to print simple values and some helpers to
format more complex structures.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:04 +02:00
David Sterba c8bea2b84b btrfs-progs: move rbtree-utils.[ch] to common/
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:04 +02:00
David Sterba 94fced6353 btrfs-progs: build: drop kernel-lib from -I and update paths
Include the files by full path to avoid any confusion in case of
potentially duplicate names.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:04 +02:00
David Sterba c07960c8be btrfs-progs: move utils.[ch] to common/
Update include paths and remove some duplicates.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:04 +02:00
David Sterba 408cec5086 btrfs-progs: move string-table.[ch] to common/
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:03 +02:00
David Sterba 07ca08ffba btrfs-progs: move fsfeatures.[ch] to common/
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:03 +02:00
David Sterba f93b471143 btrfs-progs: move help.[ch] to common/
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:03 +02:00
David Sterba d0970a05cd btrfs-progs: move task-utils.[ch] to common/
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:03 +02:00
David Sterba d1efe50d0a btrfs-progs: move messages.[ch] to common/
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:03 +02:00
David Sterba f63f29e9e9 btrfs-progs: move internal.h to common/
Create directory for all sources that can be used by anything that's not
rellated to a relevant kernel part, all common functions, helpers,
utilities that do not fit any other specific category.

The traditional location would be probably lib/ with all things that are
statically linked to the main binaries, but we have libbtrfs and
libbtrfsutil so this would be confusing.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:03 +02:00
David Sterba 96f58629c6 btrfs-progs: move cmds-send.c to cmds/
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:03 +02:00
David Sterba 4f398726d3 btrfs-progs: move all cmds-fi*.c to cmds/
This is the filesystem command group.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:03 +02:00
David Sterba 3bfc7bcfba btrfs-progs: move all cmds-inspect*.c to cmds/
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:03 +02:00
David Sterba d8bdd9e6a7 btrfs-progs: move cmds-subvolume.c to cmds/
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:02 +02:00
David Sterba 4707a1c3b3 btrfs-progs: move cmds-scrub.c to cmds/
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:02 +02:00
David Sterba fd88267f71 btrfs-progs: move cmds-receive.c to cmds/
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:02 +02:00
David Sterba f40f752bf6 btrfs-progs: move cmds-balance.c to cmds/
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:02 +02:00
David Sterba 4bee9116bb btrfs-progs: move cmds-restore.c to cmds/
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:02 +02:00
David Sterba de6b03ca33 btrfs-progs: move cmds-device.c to cmds/
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:02 +02:00
David Sterba 87caad49fa btrfs-progs: move cmds-replace.c to cmds/
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:02 +02:00
David Sterba 4356aa5f34 btrfs-progs: move and rename super-recover.c cmds/
The file now gets the prefix of the command group.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:02 +02:00
David Sterba b0445ccf0a btrfs-progs: move and rename chunk-recover.c cmds/
The file now gets the prefix of the command group.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:02 +02:00