Commit graph

133174 commits

Author SHA1 Message Date
bors 361c4ea224 Auto merge of #79049 - tmiasko:lower-intrinsics, r=jonas-schievink
Lower intrinsics calls: forget, size_of, unreachable, wrapping_*

This allows constant propagation to evaluate `size_of` and `wrapping_*`,
and unreachable propagation to propagate a call to `unreachable`.

The lowering is performed as a MIR optimization, rather than during MIR
building to preserve the special status of intrinsics with respect to
unsafety checks and promotion.

Currently enabled by default to determine the performance impact (no
significant impact expected). In practice only useful when combined with
inlining since intrinsics are rarely used directly (with exception of
`unreachable` and `discriminant_value` used by built-in derive macros).

Closes #32716.
2020-11-14 22:05:54 +00:00
bors 98d66340d6 Auto merge of #78809 - vn-ki:fix-issue-76064, r=oli-obk
add error_occured field to ConstQualifs,

fix #76064

I wasn't sure what `in_return_place` actually did and not sure why it returns `ConstQualifs` while it's sibling functions return `bool`. So I tried to make as minimal changes to the structure as possible. Please point out whether I have to refactor it or not.

r? `@oli-obk`
cc `@RalfJung`
2020-11-14 18:03:17 +00:00
bors 30e49a9ead Auto merge of #75272 - the8472:spec-copy, r=KodrAus
specialize io::copy to use copy_file_range, splice or sendfile

Fixes #74426.
Also covers #60689 but only as an optimization instead of an official API.

The specialization only covers std-owned structs so it should avoid the problems with #71091

Currently linux-only but it should be generalizable to other unix systems that have sendfile/sosplice and similar.

There is a bit of optimization potential around the syscall count. Right now it may end up doing more syscalls than the naive copy loop when doing short (<8KiB) copies between file descriptors.

The test case executes the following:

```
[pid 103776] statx(3, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_ALL, {stx_mask=STATX_ALL|STATX_MNT_ID, stx_attributes=0, stx_mode=S_IFREG|0644, stx_size=17, ...}) = 0
[pid 103776] write(4, "wxyz", 4)        = 4
[pid 103776] write(4, "iklmn", 5)       = 5
[pid 103776] copy_file_range(3, NULL, 4, NULL, 5, 0) = 5

```

0-1 `stat` calls to identify the source file type. 0 if the type can be inferred from the struct from which the FD was extracted
𝖬 `write` to drain the `BufReader`/`BufWriter` wrappers. only happen when buffers are present. 𝖬 ≾ number of wrappers present. If there is a write buffer it may absorb the read buffer contents first so only result in a single write. Vectored writes would also be an option but that would require more invasive changes to `BufWriter`.
𝖭 `copy_file_range`/`splice`/`sendfile` until file size, EOF or the byte limit from `Take` is reached. This should generally be *much* more efficient than the read-write loop and also have other benefits such as DMA offload or extent sharing.

## Benchmarks

```

OLD

test io::tests::bench_file_to_file_copy         ... bench:      21,002 ns/iter (+/- 750) = 6240 MB/s    [ext4]
test io::tests::bench_file_to_file_copy         ... bench:      35,704 ns/iter (+/- 1,108) = 3671 MB/s  [btrfs]
test io::tests::bench_file_to_socket_copy       ... bench:      57,002 ns/iter (+/- 4,205) = 2299 MB/s
test io::tests::bench_socket_pipe_socket_copy   ... bench:     142,640 ns/iter (+/- 77,851) = 918 MB/s

NEW

test io::tests::bench_file_to_file_copy         ... bench:      14,745 ns/iter (+/- 519) = 8889 MB/s    [ext4]
test io::tests::bench_file_to_file_copy         ... bench:       6,128 ns/iter (+/- 227) = 21389 MB/s   [btrfs]
test io::tests::bench_file_to_socket_copy       ... bench:      13,767 ns/iter (+/- 3,767) = 9520 MB/s
test io::tests::bench_socket_pipe_socket_copy   ... bench:      26,471 ns/iter (+/- 6,412) = 4951 MB/s
```
2020-11-14 12:01:55 +00:00
bors 66c1309446 Auto merge of #78959 - petrochenkov:likeuefi, r=nagisa
rustc_target: Mark UEFI targets as `is_like_windows`/`is_like_msvc`

And document what `is_like_windows` and `is_like_msvc` actually mean in more detail.

Addresses FIXMEs left from https://github.com/rust-lang/rust/pull/71030.
r? `@nagisa`
2020-11-14 09:11:25 +00:00
bors 1a25580c6c Auto merge of #78951 - petrochenkov:unknown, r=ehuss
rustc_target: Change os and vendor values to "none" and "unknown" for some targets

Closes https://github.com/rust-lang/rust/issues/77730
r? `@ehuss`
2020-11-14 06:44:18 +00:00
bors 50d3c2a3cb Auto merge of #78736 - petrochenkov:lazyenum, r=Aaron1011
rustc_parse: Remove optimization for 0-length streams in `collect_tokens`

The optimization conflates empty token streams with unknown token stream, which is at least suspicious, and doesn't affect performance because 0-length token streams are very rare.

r? `@Aaron1011`
2020-11-14 04:21:56 +00:00
Tomasz Miąsko 6903273339 Lower intrinsics calls: forget, size_of, unreachable, wrapping_*
This allows constant propagation to evaluate `size_of` and `wrapping_*`,
and unreachable propagation to propagate a call to `unreachable`.

The lowering is performed as a MIR optimization, rather than during MIR
building to preserve the special status of intrinsics with respect to
unsafety checks and promotion.
2020-11-14 00:00:00 +00:00
bors b63d05a908 Auto merge of #78683 - Nemo157:issue-78673, r=lcnr
Check predicates from blanket trait impls while testing if they apply

fixes #78673
2020-11-13 23:12:01 +00:00
The8472 bbfa92c82d Always handle EOVERFLOW by falling back to the generic copy loop
Previously EOVERFLOW handling was only applied for io::copy specialization
but not for fs::copy sharing the same code.

Additionally we lower the chunk size to 1GB since we have a user report
that older kernels may return EINVAL when passing 0x8000_0000
but smaller values succeed.
2020-11-13 22:38:27 +01:00
The8472 4854d418a5 do direct splice syscall and probe availability to get android builds to work
Android builds use feature level 14, the libc wrapper for splice is gated
on feature level 21+ so we have to invoke the syscall directly.
Additionally the emulator doesn't seem to support it so we also have to
add ENOSYS checks.
2020-11-13 22:38:27 +01:00
The8472 3dfc377aa1 move sendfile/splice/copy_file_range into kernel_copy module 2020-11-13 22:38:27 +01:00
The8472 888b1031bc limit visibility of copy offload helpers to sys::unix module 2020-11-13 22:38:27 +01:00
The8472 18bfe2a66b move copy specialization tests to their own module 2020-11-13 22:38:27 +01:00
The8472 7f5d2722af move copy specialization into sys::unix module 2020-11-13 22:38:23 +01:00
bors 74f7e32f43 Auto merge of #78888 - richkadel:llvm-coverage-tests, r=tmandry
Fix and re-enable two coverage tests on MacOS

Note, in the coverage-reports test, the comment about MacOS was wrong.
The setting is based on config.toml llvm `optimize` setting. There
doesn't appear to be any environment variable I can check, and I
don't think we should add one. Testing the binary itself is a more
reliable way to check anyway.

For the coverage-spanview test, I removed the dependency on sed
altogether, which is much less ugly than trying to work around the
MacOS sed differences.

I tested these changes on Linux, Windows, and Mac.

r? `@tmandry`
FYI `@wesleywiser`
2020-11-13 20:06:46 +00:00
The8472 ad9b07c7e5 add benchmarks 2020-11-13 19:46:37 +01:00
The8472 46e7fbe60b reduce syscalls by inferring FD types based on source struct instead of calling stat()
also adds handling for edge-cases involving large sparse files where sendfile could fail with EOVERFLOW
2020-11-13 19:46:35 +01:00
The8472 0624730d9e add forwarding specializations for &mut variants
`impl Write for &mut T where T: Write`, thus the same should
apply to the specialization traits
2020-11-13 19:45:38 +01:00
The8472 cd3bddc044 prioritize sendfile over splice since it results in fewer context switches when sending to pipes
splice returns to userspace when the pipe is full, sendfile
just blocks until it's done, this can achieve much higher throughput
2020-11-13 19:45:38 +01:00
The8472 67a6059aa5 move tests module into separate file 2020-11-13 19:45:38 +01:00
The8472 5eb88fa5c7 hide unused exports on other platforms 2020-11-13 19:45:38 +01:00
The8472 16236470c1 specialize io::copy to use copy_file_range, splice or sendfile
Currently it only applies to linux systems. It can be extended to make use
of similar syscalls on other unix systems.
2020-11-13 19:45:27 +01:00
bors 89bce3e908 Auto merge of #79011 - ehuss:update-cargo, r=ehuss
Update cargo

Fixing an important publish bug.

2 commits in 8662ab427a8d6ad8047811cc4d78dbd20dd07699..2af662e22177a839763ac8fb70d245a680b15214
2020-11-12 03:47:53 +0000 to 2020-11-12 19:04:56 +0000
- Fix publishing with optional dependencies. (rust-lang/cargo#8853)
- Minor typo in features.md (rust-lang/cargo#8851)
2020-11-13 17:35:56 +00:00
bors f2a11a2537 Auto merge of #79017 - GuillaumeGomez:rollup-5orhudd, r=GuillaumeGomez
Rollup of 6 pull requests

Successful merges:

 - #77151 (Add regression test for issue #76042)
 - #77996 (Doc change: Remove mention of `fnv` in HashMap)
 - #78463 (Add type to `ConstKind::Placeholder`)
 - #78984 (Rustdoc check option)
 - #78985 (add dropck test for const params)
 - #78996 (add explicit test for const param promotion)

Failed merges:

r? `@ghost`
`@rustbot` modify labels: rollup
2020-11-13 14:53:55 +00:00
Guillaume Gomez 7ea8e32adb
Rollup merge of #78996 - lcnr:cg-promotion, r=RalfJung
add explicit test for const param promotion

r? `@RalfJung`
2020-11-13 15:26:20 +01:00
Guillaume Gomez 22cc878d89
Rollup merge of #78985 - lcnr:cg-drop-test, r=nikomatsakis
add dropck test for const params

r? `@nikomatsakis` or `@varkor`
2020-11-13 15:26:18 +01:00
Guillaume Gomez 804ca1d5b7
Rollup merge of #78984 - GuillaumeGomez:rustdoc-check-option, r=jyn514
Rustdoc check option

The ultimate goal behind this option would be to have `rustdoc --check` being run when you use `cargo check` as a second step.

r? `@jyn514`
2020-11-13 15:26:16 +01:00
Guillaume Gomez 0b7a7930b4
Rollup merge of #78463 - varkor:placeholder-const, r=nikomatsakis
Add type to `ConstKind::Placeholder`

I simply threaded `<'tcx>` through everything that required it. I'm not sure whether this is the correct thing to do, but it seems to work.

r? `@nikomatsakis`
2020-11-13 15:26:14 +01:00
Guillaume Gomez ef32ef7baf
Rollup merge of #77996 - tkaitchuck:master, r=m-ou-se
Doc change: Remove mention of `fnv` in HashMap

Disclaimer: I am the author of [aHash](https://github.com/tkaitchuck/aHash).

This changes the Rustdoc in `HashMap` from mentioning the `fnv` crate to mentioning the `aHash` crate, as an alternative `Hasher` implementation.

### Why

Fnv [has poor hash quality](https://github.com/rurban/smhasher), is [slow for larger keys](https://github.com/tkaitchuck/aHash/blob/master/compare/readme.md#speed), and does not provide dos resistance, because it is unkeyed (this can also cause [other problems](https://accidentallyquadratic.tumblr.com/post/153545455987/rust-hash-iteration-reinsertion)).

Fnv has acceptable performance for integers and has very poor performance with keys >32 bytes. This is the reason it was removed from the standard library in https://github.com/rust-lang/rust/pull/37229 .

Because regardless of which dimension you value, there are better alternatives, it does not make sense for anyone to consider using `fnv`.

The text mentioning `fnv` in the standard library continues to create confusion: https://github.com/rust-lang/hashbrown/issues/153  https://github.com/rust-lang/hashbrown/issues/9 . There are also a number of [crates using it](https://crates.io/crates/fnv/reverse_dependencies) a great many of which are hashing strings (Which is when Fnv is the [worst](https://github.com/cbreeden/fxhash#benchmarks), [possible](https://github.com/tkaitchuck/aHash#speed), [choice](http://cglab.ca/~abeinges/blah/hash-rs/).)

I think aHash makes the most sense to mention as an alternative because it is the most credible option (in my obviously biased opinion). It offers [good performance on numbers and strings](https://github.com/tkaitchuck/aHash/blob/master/compare/readme.md#speed), is [of high quality](https://github.com/tkaitchuck/aHash#hash-quality), and [provides dos resistance](https://github.com/tkaitchuck/aHash/wiki/How-aHash-is-resists-DOS-attacks). It is popular (see [stats](https://crates.io/crates/ahash)) and is the default hasher for [hashbrown](https://crates.io/crates/hashbrown) and [dashmap](https://crates.io/crates/dashmap) which are the most popular alternative hashmaps. Finally it does not have any of the [`gotcha` cases](https://github.com/tkaitchuck/aHash#fxhash) that `FxHash` suffers from. (Which is the other popular hashing option when DOS attacks are not a concern)

Signed-off-by: Tom Kaitchuck <tom.kaitchuck@emc.com>
2020-11-13 15:26:10 +01:00
Guillaume Gomez c5a11ddec9
Rollup merge of #77151 - rust-lang:LeSeulArtichaut-patch-1, r=pnkfelix
Add regression test for issue #76042

Originally posted in https://github.com/rust-lang/rust/issues/76042#issuecomment-683272710.
r? `@pnkfelix`
2020-11-13 15:26:06 +01:00
bors a1f7ca788d Auto merge of #77990 - bugadani:copies, r=lcnr
Eliminate some temporary vectors

This PR changes `get_item_attrs` and `get_item_variances` to return iterator impls instead of vectors. On top of that, this PR replaces some seemingly unnecessary vectors with iterators or SmallVec, and also reserves space where we know (the minimum) number of elements that will be inserted. This change hopes to remove a few heap allocations and unnecessary copies.
2020-11-13 11:49:33 +00:00
Vishnunarayan K I f026d0fcf8 update clippy test ouput 2020-11-13 17:11:13 +05:30
Dániel Buga f0d0d87a20 Push to result vector instead of allocating
Co-authored-by: lcnr <bastian_kauschke@hotmail.de>
2020-11-13 11:19:25 +01:00
Dániel Buga 45faeb43ae Reuse vector 2020-11-13 11:19:25 +01:00
Dániel Buga a7f2bb6343 Reserve space in advance 2020-11-13 11:19:25 +01:00
Dániel Buga 2ca0b85593 Allocate less in lower_block_noalloc 2020-11-13 11:19:25 +01:00
Dániel Buga 660d8a6550 Eliminate some temporary vectors & Remove unnecessary mark_attr_used 2020-11-13 11:19:23 +01:00
bors 2c4df6bb34 Auto merge of #79014 - pietroalbini:1.50, r=pietroalbini
Bump version number to 1.50.0

First PR of the release process of Rust 1.48.0. All PRs landed after this one will be included in Rust 1.50.0.

r? `@ghost`
cc `@rust-lang/release`
2020-11-13 09:07:29 +00:00
Pietro Albini 2306e6277c
bump version to 1.50.0 2020-11-13 09:47:16 +01:00
bors a38f8fb674 Auto merge of #78826 - petrochenkov:mrscopes2, r=eddyb
resolve: Collapse `macro_rules` scope chains on the fly

Otherwise they grow too long and you have to endlessly walk through them when resolving macros or imports.
Addresses https://rust-lang.zulipchat.com/#narrow/stream/247081-t-compiler.2Fperformance/topic/Slow.20Builtin.20Derives/near/215750815
2020-11-13 05:40:37 +00:00
Tom Kaitchuck 4e5848349c
Update library/std/src/collections/hash/map.rs
Co-authored-by: Mara Bos <m-ou.se@m-ou.se>
2020-11-12 20:14:57 -08:00
Eric Huss a830720098 Update cargo 2020-11-12 18:39:16 -08:00
bors f036a8f3be Auto merge of #78990 - RalfJung:miri, r=RalfJung
update Miri

Fixes https://github.com/rust-lang/rust/issues/78937
Cc `@rust-lang/miri` r? `@ghost`
2020-11-13 02:04:53 +00:00
bors e80ee05bfc Auto merge of #78998 - m-ou-se:rollup-6r4pt9m, r=m-ou-se
Rollup of 7 pull requests

Successful merges:

 - #76730 (Fix rustdoc rendering of by-value mutable arguments in async fn)
 - #78836 (Implement destructuring assignment for structs and slices)
 - #78857 (Improve BinaryHeap performance)
 - #78950 (Add asm register information for SPIR-V)
 - #78970 (update rustfmt to v1.4.25)
 - #78972 (Update cargo)
 - #78987 (extend min_const_generics param ty tests)

Failed merges:

r? `@ghost`
`@rustbot` modify labels: rollup
2020-11-12 23:23:56 +00:00
Guillaume Gomez a06fd1f4f4 Ensure that INVALID_CODEBLOCK_ATTRIBUTES lint is emitted 2020-11-12 22:41:10 +01:00
Vadim Petrochenkov 2879ab793e rustc_parse: Remove optimization for 0-length streams in collect_tokens
The optimization conflates empty token streams with unknown token stream, which is at least suspicious, and doesn't affect performance because 0-length token streams are very rare.
2020-11-12 22:00:48 +03:00
Mara Bos 38ca6e3561
Rollup merge of #78987 - lcnr:integer-sizes, r=varkor
extend min_const_generics param ty tests

Apparently we never tested for `u128` and `i128` before this, so I added a test for all types which are allowed.

r? ``@varkor``
2020-11-12 19:46:19 +01:00
Mara Bos ef77a43402
Rollup merge of #78972 - ehuss:update-cargo, r=ehuss
Update cargo

5 commits in d5556aeb8405b1fe696adb6e297ad7a1f2989b62..8662ab427a8d6ad8047811cc4d78dbd20dd07699
2020-11-04 22:20:36 +0000 to 2020-11-12 03:47:53 +0000
- Check if rust-src contains a vendor dir, and patch it in (rust-lang/cargo#8834)
- Improve performance of almost fresh builds (rust-lang/cargo#8837)
- Use u32/64::to/from_le_bytes instead of bit fiddling (rust-lang/cargo#8847)
- Avoid constructing an anyhow::Error when not necessary (rust-lang/cargo#8844)
- Skip extracting .cargo-ok files from packages (rust-lang/cargo#8835)
2020-11-12 19:46:17 +01:00
Mara Bos a2e8fb540d
Rollup merge of #78970 - calebcartwright:update-rustfmt, r=Aaron1011
update rustfmt to v1.4.25

Contains changes from https://github.com/rust-lang/rustfmt/pull/4507

r? ``@Aaron1011``
2020-11-12 19:46:16 +01:00
Mara Bos 76fa5f25ab
Rollup merge of #78950 - khyperia:spirv-asm, r=Amanieu
Add asm register information for SPIR-V

As discussed in [zulip](https://rust-lang.zulipchat.com/#narrow/stream/182449-t-compiler.2Fhelp/topic/Defining.20asm!.20for.20new.20architecture), we at [rust-gpu](https://github.com/EmbarkStudios/rust-gpu) would like to support `asm!` for our SPIR-V backend. However, we cannot do so purely without frontend support: [this match](d4ea0b3e46/compiler/rustc_target/src/asm/mod.rs (L185)) fails and so `asm!` is not supported ([error reported here](d4ea0b3e46/compiler/rustc_ast_lowering/src/expr.rs (L1095))). To resolve this, we need to stub out register information for SPIR-V to support getting the `asm!` content all the way to [`AsmBuilderMethods::codegen_inline_asm`](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_codegen_ssa/traits/trait.AsmBuilderMethods.html#tymethod.codegen_inline_asm), at which point the rust-gpu backend can do all the parsing and codegen that is needed.

This is a pretty weird PR - adding support for a backend that isn't in-tree feels pretty gross to me, but I don't see an easy way around this. ``@Amanieu`` said I should submit it anyway, so, here we are! Let me know if this needs to go through a more formal process (MCP?) and what I should do to help this along.

I based this off the [wasm asm PR](https://github.com/rust-lang/rust/pull/78684), which unfortunately this PR conflicts with that one quite a bit, sorry for any merge conflict pain :(

---

Some open questions:

- What do we call the register class? Some context, SPIR-V is an SSA-based IR, there are "instructions" that create IDs (referred to as `<id>` in the spec), which can be referenced by other instructions. So, `reg` isn't exactly accurate, they're SSA IDs, not re-assignable registers.
- What happens when a SPIR-V register gets to the LLVM backend? Right now it's a `bug!`, but should that be a `sess.fatal()`? I'm not sure if it's even possible to reach that point, maybe there's a check that prevents the `spirv` target from even reaching that codepath.
2020-11-12 19:46:14 +01:00