rust/library
bors 30e49a9ead Auto merge of #75272 - the8472:spec-copy, r=KodrAus
specialize io::copy to use copy_file_range, splice or sendfile

Fixes #74426.
Also covers #60689 but only as an optimization instead of an official API.

The specialization only covers std-owned structs so it should avoid the problems with #71091

Currently linux-only but it should be generalizable to other unix systems that have sendfile/sosplice and similar.

There is a bit of optimization potential around the syscall count. Right now it may end up doing more syscalls than the naive copy loop when doing short (<8KiB) copies between file descriptors.

The test case executes the following:

```
[pid 103776] statx(3, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_ALL, {stx_mask=STATX_ALL|STATX_MNT_ID, stx_attributes=0, stx_mode=S_IFREG|0644, stx_size=17, ...}) = 0
[pid 103776] write(4, "wxyz", 4)        = 4
[pid 103776] write(4, "iklmn", 5)       = 5
[pid 103776] copy_file_range(3, NULL, 4, NULL, 5, 0) = 5

```

0-1 `stat` calls to identify the source file type. 0 if the type can be inferred from the struct from which the FD was extracted
𝖬 `write` to drain the `BufReader`/`BufWriter` wrappers. only happen when buffers are present. 𝖬 ≾ number of wrappers present. If there is a write buffer it may absorb the read buffer contents first so only result in a single write. Vectored writes would also be an option but that would require more invasive changes to `BufWriter`.
𝖭 `copy_file_range`/`splice`/`sendfile` until file size, EOF or the byte limit from `Take` is reached. This should generally be *much* more efficient than the read-write loop and also have other benefits such as DMA offload or extent sharing.

## Benchmarks

```

OLD

test io::tests::bench_file_to_file_copy         ... bench:      21,002 ns/iter (+/- 750) = 6240 MB/s    [ext4]
test io::tests::bench_file_to_file_copy         ... bench:      35,704 ns/iter (+/- 1,108) = 3671 MB/s  [btrfs]
test io::tests::bench_file_to_socket_copy       ... bench:      57,002 ns/iter (+/- 4,205) = 2299 MB/s
test io::tests::bench_socket_pipe_socket_copy   ... bench:     142,640 ns/iter (+/- 77,851) = 918 MB/s

NEW

test io::tests::bench_file_to_file_copy         ... bench:      14,745 ns/iter (+/- 519) = 8889 MB/s    [ext4]
test io::tests::bench_file_to_file_copy         ... bench:       6,128 ns/iter (+/- 227) = 21389 MB/s   [btrfs]
test io::tests::bench_file_to_socket_copy       ... bench:      13,767 ns/iter (+/- 3,767) = 9520 MB/s
test io::tests::bench_socket_pipe_socket_copy   ... bench:      26,471 ns/iter (+/- 6,412) = 4951 MB/s
```
2020-11-14 12:01:55 +00:00
..
alloc Rollup merge of #78857 - SkiFire13:bheap-opt, r=KodrAus 2020-11-12 19:46:11 +01:00
backtrace@8b8ea53b56 Bump backtrace-rs to enable Mach-O support on iOS. 2020-10-23 13:47:09 +02:00
core Rollup merge of #78216 - workingjubilee:duration-zero, r=m-ou-se 2020-11-11 20:58:52 +01:00
panic_abort Rollup merge of #76866 - est31:master, r=lcnr 2020-09-20 15:51:50 +02:00
panic_unwind Fix even more URLs 2020-11-05 20:11:29 +01:00
proc_macro Rollup merge of #78465 - est31:proc_macro_to_string, r=jyn514 2020-11-05 10:29:42 +01:00
profiler_builtins Fix warning whe building profiler_builtins crate 2020-09-04 15:10:29 +02:00
rtstartup
rustc-std-workspace-alloc
rustc-std-workspace-core
rustc-std-workspace-std
std Auto merge of #75272 - the8472:spec-copy, r=KodrAus 2020-11-14 12:01:55 +00:00
stdarch@3c3664355e Update stdarch submodule 2020-10-01 13:06:22 -04:00
term
test Rollup merge of #78065 - tshepang:nits, r=dtolnay 2020-11-08 13:36:01 +01:00
unwind allow using the system-wide llvm-libunwind as the unwinder 2020-10-21 14:45:58 +02:00