NLL: improve inference with flow results, represent regions with bitsets, and more
This PR begins with a number of edits to the NLL code and then includes a large number of smaller refactorings (these refactorings ought not to change behavior). There are a lot of commits here, but each is individually simple. The goal is to land everything up to but not including the changes to how we handle closures, which are conceptually more complex.
The NLL specific changes are as follows (in order of appearance):
**Modify the region inferencer's approach to free regions.** Previously, for each free region (lifetime parameter) `'a`, it would compute the set of other free regions that `'a` outlives (e.g., if we have `where 'a: 'b`, then this set would be `{'a, 'b}`). Then it would mark those free regions as "constants" and report an error if inference tried to extend `'a` to include any other region (e.g., `'c`) that is not in that outlives set. In this way, the value of `'a` would never grow beyond the maximum that could type check. The new approach is to allow `'a` to grow larger. Then, after the fact, we check over the value of `'a` and see what other free regions it is required to outlive, and we check that those outlives relationships are justified by the where clauses in scope etc.
**Modify constraint generation to consider maybe-init.** When we have a "drop-live" variable `x` (i.e., a variable that will be dropped but will not be otherwise used), we now consider whether `x` is "maybe initialized" at that point. If not, then we know the drop is a no-op, and we can allow its regions to be dead. Due to limitations in the fragment code, this currently only works at the level of entire variables.
**Change representation of regions to use a `BitMatrix`.** We used to use a `BTreeSet`, which was rather silly. We now use a MxN matrix of bits, where `M` is the number of variables and `N` is the number of possible elements in each set (size of the CFG + number of free regions).
The remaining commits (starting from
extract the `implied_bounds` code into a helper function ") are all "no-op" refactorings, I believe.
~~One concern I have is with the commit "with -Zverbose, print all details of closure substs"; this commit seems to include some "internal" stuff in the mir-dump files, such as internal interner numbers, that I fear may vary by platform. Annoying. I guess we will see.~~ (I removed this commit.)
As for reviewer, @arielb1 has been reviewing the PRs, and they are certainly welcome to review this one too. But I figured it'd maybe be good to have more people taking a look and being familiar with this code, so I'll "nominate" @pnkfelix .
r? @pnkfelix
Fix invalid docs path for compiler plugins
The path to the docs `src/doc/guide-plugin.md` moved to
`src/doc/unstable-book/src/language-features/plugin.md`.
This patch updates it in the comment of WARNING message of the test
code.
This has been bugging me. All the regions appear free in the source;
the real difference is that some of them are universally quantified
(those in the function signature) and some are existentially
quantified (those for which we are inferring values).
We now visit just the stuff in the CFG, and we add liveness
constraints for all the random types, regions etc that appear within
rvalues and statements.
In particular, if we see a variable is DROP-LIVE, but it is not
MAYBE-INIT, then we can ignore the drop. This leavess attempt to use
more complex refinements of the idea (e.g., for subpaths or subfields)
to future work.
Rather than declaring some region variables to be constant, and
reporting errors when they would have to change, we instead populate
each free region X with a minimal set of points (the CFG plus end(X)),
and then we let inference do its thing. This may add other `end(Y)`
points into X; we can then check after the fact that indeed `X: Y`
holds.
This requires a bit of "blame" detection to find where the bad
constraint came from: we are currently using a pretty dumb
algorithm. Good place for later expansion.
Add a specialization of read_exact for Cursor.
The read_exact implementation for &[u8] is optimized and usually allows LLVM to reduce a read_exact call for small numbers of bytes to a bounds check and a register load instead of a generic memcpy. On a workload I have that decompresses, deserializes (via bincode), and processes some data, this leads to a 40% speedup by essentially eliminating the deserialization overhead entirely.
The read_exact implementation for &[u8] is optimized and usually allows LLVM to reduce a read_exact call for small numbers of bytes to a bounds check and a register load instead of a generic memcpy. On a workload I have that decompresses, deserializes (via bincode), and processes some data, this leads to a 40% speedup by essentially eliminating the deserialization overhead entirely.
Consistent parameter name for numeric ‘checked’ operations.
Some checked operations use `rhs` as a parameter name, and some use
`other`. For the sake of consistency, unify everything under the `rhs`
name.
Fixes https://github.com/rust-lang/rust/issues/46308.
Fix invalid link to lint_plugin_test.rs
The path to `lint_plugin_test.rs` was moved to `src/test/ui-fulldeps/`
from `src/test/run-pass-fulldeps/` in 38ef85696d
This patch updates it in the docs.
Fix CopyPropagation regression (2)
Remaining part of MIR copyprop regression by (I think) #45380, which I missed in #45753.
```rust
fn foo(mut x: i32) -> i32 {
let y = x;
x = 123; // `x` is assigned only once in MIR, but cannot be propagated to `y`
y
}
```
So any assignment to an argument cannot be propagated.
Make doc stubs for builtin macros reflect existing support for trailing commas
This modifies the `macro_rules!` stubs in `std` for some of the compiler builtin macros in order to better reflect their currently supported grammar. To my understanding these stubs have no impact on compiler output whatsoever, and only exist so that they may appear in the documentation.
P.S. It is in fact true that `env!` supports trailing commas while `option_env!` currently does not. (I have another issue for this)
I don't imagine there's any way to automatically test these stubs, but I did *informally* test the new definitions on the playpen to see that they accept the desired invocations, as well as inspect the updated doc output.
Use more convenient and UNIX-agnostic shebang
When using bash-specific features, scripts using env to call bash
are more convenient, as bash be installed in different places
according the OS.
Assume at least LLVM 3.9 in rustllvm and rustc_llvm
We bumped the minimum LLVM to 3.9 in #45326. This just cleans up the conditional code in the `rustllvm` C++ wrappers to assume that minimum, and similarly cleans up the `rustc_llvm` build script.
[auto-toolstate][2+3/8] Move external tools tests into its own job with --no-fail-fast
This PR performs these things:
1. The `aux` job now performs "cargotest" and "pretty" tests. The clippy/rustfmt/rls/miri tests are moved into its own job.
2. These tests are run with `--no-fail-fast`, so that we can get the maximum number of failures of all tools from a single CI run.
3. The test results are stored into a JSON file, ready to be uploaded in the future.
This is step 2 and 3/8 of automatic management of broken tools #45861.
Some checked operations use `rhs` as a parameter name, and some use
`other`. For the sake of consistency, unify everything under the `rhs`
name.
Fixes https://github.com/rust-lang/rust/issues/46308.
create a drop ladder for an array if any value is moved out
r? @arielb1
first commit for fix https://github.com/rust-lang/rust/issues/34708 (note: this still handles the subslice case in a very broken manner)
rustc: don't unpack newtypes of scalar-pairs with mismatched alignment.
This PR fixes a potential problem where a packed newtype of a pair was also considered a pair, even though it didn't have the required alignment of the pair.
cc @oli-obk It's possible miri hit something like this, with an unstable feature, but it's more general.
rustdoc: Fix issues with cross-crate inlined associated items
* Visibility was missing from impl items.
* Attributes and docs were missing from consts and types in impls.
* Const default values were missing from traits.
This unifies the code that handles associated items from impls and traits.
This simplifies analysis and borrow-checking because liveness at the
resume point can always be simply propagated.
Later on, the "dead" Resumes are removed.
Region inference can create borrows for an empty region if the borrow is
dead. In that case, there's no reason to track the borrow, but because
there's no such thing as an EndRegion(ReEmpty) these borrows used to live
for the entire function.
Fixes#46161.
rustc: Prepare to enable ThinLTO by default
This commit *almost* enables ThinLTO and multiple codegen units in release mode by
default but is blocked on #46346 now before pulling the trigger.
On type mismatch error highlight `&` when type matches
When the only difference between the two types in a type error is that
one is a reference to the other type (`T` vs `&T`) or both are
references differing only in their mutability (`&T` vs `&mut T`), don't
highlight the type (`T`).
Add case insensitive comparison, besides Levenstein for DYM
Closes#46332
Draft version. The idea is that Levenstein does not work for some cases when we have multiple equal weights for strings. I didn't understand the case with `if found != name => Some(found)` so it means that new code does not work correctly yet.
At least now I think that we might return all maximal weights from levenstein and think about next cases in priority order:
1) There is exact match -> None
2) There is exact match, but case insensitive -> Some(match)
3) There is some match from levenstein -> Some(matches.take_any)
4) There is no match -> None
@estebank WDYT?
Use suggestions instead of notes ref mismatches
On type mismatch errors, use a suggestion when encountering minimal
differences in type differences due to refs, instead of a note.
When using bash-specific features, scripts using env to call bash
are more convenient, as bash be installed in different places
according the OS.
Same applies for other languages' interpreters.
Generic Associated Types Parsing & Name Resolution
Hi!
This PR adds parsing for generic associated types! 🎉🎉🎉
Tracking Issue: #44265
## Notes For Reviewers
* [x] I still need to add the stdout and stderr files to my ui tests. It takes me a *long* time to compile the compiler locally, so I'm going to add this as soon as possible in the next day or so.
* [ ] My current ui tests aren't very good or very thorough. I'm reusing the `parse_generics` and `parse_where_clause` methods from elsewhere in the parser, so my changes work without being particularly complex. I'm not sure if I should duplicate all of the generics test cases for generic associated types. It might actually be appropriate to duplicate everything here, since we don't want to rely on an implementation detail in case it changes in the future. If you think so too, I'll adapt all of the generics test cases into the generic associated types test cases.
* [ ] There is still more work required to make the run-pass tests pass here. In particular, we need to make the following errors disappear:
```
error[E0110]: lifetime parameters are not allowed on this type
--> ./src/test/run-pass/rfc1598-generic-associated-types/streaming_iterator.rs:23:41
|
23 | bar: <T as StreamingIterator>::Item<'static>,
| ^^^^^^^ lifetime parameter not allowed on this type
```
```
error[E0261]: use of undeclared lifetime name `'a`
--> ./src/test/run-pass/rfc1598-generic-associated-types/iterable.rs:15:47
|
15 | type Iter<'a>: Iterator<Item = Self::Item<'a>>;
| ^^ undeclared lifetime
```
There is a FIXME comment in streaming_iterator. If you uncomment that line, you get the following:
```
error: expected one of `!`, `+`, `,`, `::`, or `>`, found `=`
--> ./src/test/run-pass/rfc1598-generic-associated-types/streaming_iterator.rs:29:45
|
29 | fn foo<T: for<'a> StreamingIterator<Item<'a>=&'a [i32]>>(iter: T) { /* ... */ }
| ^ expected one of `!`, `+`, `,`, `::`, or `>` here
```
r? @nikomatsakis
MIR: change "lvalue" terminology to "place".
As pointed out elsewhere, "lvalue" vs "rvalue" is a misleading/obscure distinction and several other choices have been proposed, the one I prefer being "place" vs "value".
This PR only touches the "lvalue" side, and only in MIR-related code, as it's already a lot and could rot.
build_helper: destination file can't be up to date when not exists
Function "up_to_date" return incorrect result if mtime for all fetched sources is set to epoch time. Add existence check to function.
This fix required for a [Guix](https://www.gnu.org/software/guix/) package because a Nix builder set mtime of all sources to epoch time.
Hide private trait type params and show hidden items with document-private
As discussed in #46380, this PR removes the `strip-hidden` pass from `--document-private-items` which allows showing `#[doc(hidden)]` with rustdoc.
The second commit removes the trait implementation from the docs if the trait's parameter is private.
rustc: Filter out bogus extern crate warnings
Rustdoc has for some time now used the "everybody loops" pass in the compiler to
avoid typechecking and otherwise avoid looking at implementation details.
In #46115 the placement of this pass was pushed back in the compiler to after
macro expansion to ensure that it works with macro-expanded code as well. This
in turn caused the regression in #46271.
The bug here was that the resolver was producing `def_id` instances for
"possibly unused extern crates" which would then later get processed during
typeck to actually issue lint warnings. The problem was that *after* resolution
these `def_id` nodes were actually removed from the AST by the "everybody loops"
pass. This later, when we tried to take a look at `def_id`, caused the compiler
to panic.
The fix applied here is a bit of a heavy hammer which is to just, in this one
case, ignore the `extern crate` lints if the `def_id` looks "bogus" in any way
(basically if it looks like the node was removed after resolution). The real
underlying bug here is probably that the "everybody loops" AST pass is being
stressed to much beyond what it was originally intended to do, but this should
at least fix the ICE for now...
Closes#46271
wasm: Update LLVM to fix a test
This commit updates LLVM with some tweaks to the integer <-> floating point
conversion instructions to ensure that `as` in Rust doesn't trap.
Closes#46298
incr.comp.: Make traits::VTable encodable and decodable.
Make vtables encodable so we can cache the `trans_fulfill_obligation` query at some point.
r? @eddyb
Fix rustdoc item summaries that are headers
Rustoc item summaries that are headers were not displayed at all because
they started with whitespace.
This PR fixes this and now removes the whitespace and then displays the
block.
I'm not sure if the rustdoc test is written correctly, if there's anything to improve, just let me know. :)
This fixes#46377.
This is how it looks when rendered out now:
Remove librustdoc dependency on env_logger
We want librustdoc to pickup the env_logger dependency from
the sysroot. This ensures that the same copy of env_logger is used
for both internal crates (e.g. librustc_driver, libsyntax) and
librustdoc
Closes#46383
rustbuild: Fix a typo with the Cargo book
The usage of `Path::new` prevented out-of-tree builds (like the bots do) from
working by accident!
Closes#46195
NetBSD: add sysctl backend for std::env::current_exe
Use the CTL_KERN.KERN_PROC_ARGS.-1.KERN_PROC_PATHNAME sysctl in
preference over the /proc/curproc/exe symlink.
Additionally, perform more validation of aformentioned symlink.
Particularly on pre-8.x NetBSD this symlink will point to '/' when
accurate information is unavailable.
make coercions to `!` in unreachable code a hard error
This was added to cover up a lazy extra semicolon in #35849, but does
not actually make sense. This is removed as a part of the stabilization
of `never_type`.
incr.comp.: Load cached diagnostics lazily and allow more things in the cache.
This PR implements makes two changes:
1. Diagnostics are loaded lazily from the incr. comp. cache now. This turned out to be necessary for correctness because diagnostics contain `Span` values and deserializing those requires that the source file they point to is still around in the current compilation session. Obviously this isn't always the case. Loading them lazily allows for never touching diagnostics that are not valid anymore.
2. The compiler can now deal with there being no cache entry for a given query invocation. Before, all query results of a cacheable query were always expected to be present in the cache. Now, the compiler can fall back to re-computing the result if there is no cache entry found. This allows for caching things that we cannot force from dep-node (like the `symbol_name` query). In such a case we'll just have a "best effort" caching strategy.
~~This PR is based on https://github.com/rust-lang/rust/pull/46301 (=first 2 commits), so please don't merge until that has landed. The rest of the commits are ready for review though.~~
r? @nikomatsakis
The previous method ran into problems because ICH would treat Spans
as (file,line,col) but the cache contained byte offsets and its
possible for the latter to change while the former stayed stable.
incr.comp.: Remove ability to produce incr. comp. hashes during metadata export.
This functionality has been superseded by on-import hashing, which can be less conservative and does not require extra infrastructure.
r? @nikomatsakis
After renaming the structs and enums the htmldocck strings still
contained the old names. This lead to test failure.
These htmldocck tests have been updated to use the proper names of the
rust structs and traits.
Trait's implementations with private type parameters were displayed in
the implementing struct's documentation until now.
With this change any trait implementation that uses a private type
parameter is now hidden in the docs.
When using `#[doc(hidden)]` elements are hidden from docs even when the
rustdoc flag `--document-private-items` is set.
This behavior has been changed to display all hidden items when the flag
is active.
Use the CTL_KERN.KERN_PROC_ARGS.-1.KERN_PROC_PATHNAME sysctl in
preference over the /proc/curproc/exe symlink.
Additionally, perform more validation of aformentioned symlink.
Particularly on pre-8.x NetBSD this symlink will point to '/' when
accurate information is unavailable.
Rustdoc has for some time now used the "everybody loops" pass in the compiler to
avoid typechecking and otherwise avoid looking at implementation details.
In #46115 the placement of this pass was pushed back in the compiler to after
macro expansion to ensure that it works with macro-expanded code as well. This
in turn caused the regression in #46271.
The bug here was that the resolver was producing `def_id` instances for
"possibly unused extern crates" which would then later get processed during
typeck to actually issue lint warnings. The problem was that *after* resolution
these `def_id` nodes were actually removed from the AST by the "everybody loops"
pass. This later, when we tried to take a look at `def_id`, caused the compiler
to panic.
The fix applied here is a bit of a heavy hammer which is to just, in this one
case, ignore the `extern crate` lints if the `def_id` looks "bogus" in any way
(basically if it looks like the node was removed after resolution). The real
underlying bug here is probably that the "everybody loops" AST pass is being
stressed to much beyond what it was originally intended to do, but this should
at least fix the ICE for now...
Closes#46271
This commit prepares to enable ThinLTO and multiple codegen units in release
mode by default. We've still got a debuginfo bug or two to sort out before
actually turning it on by default.
Previously we were too eagerly exporting almost all symbols used in ThinLTO
which can cause a whole host of problems downstream! This commit instead fixes
this error by aligning more closely with `lib/LTO/LTO.cpp` in LLVM's codebase
which is to only change the linkage of summaries which are computed as dead.
Closes#46374
This commit updates LLVM with some tweaks to the integer <-> floating point
conversion instructions to ensure that `as` in Rust doesn't trap.
Closes#46298
This was added to cover up a lazy extra semicolon in #35849, but does
not actually make sense. This is removed as a part of the stabilization
of `never_type`.
incr.comp.: Some preparatory work for caching more query results.
This PR
* adds and updates some encoding/decoding routines for various query result types so they can be cached later, and
* adds missing `[input]` annotations for a few `DepNode` variants.
The situation around having to explicitly mark dep-nodes/queries as inputs is not really satisfactory. I hope we can find a way of making this more fool-proof in the future.
r? @nikomatsakis
avoid type-live-for-region obligations on dummy nodes
Type-live-for-region obligations on DUMMY_NODE_ID cause an ICE, and it
turns out that in the few cases they are needed, these obligations are not
needed anyway because they are verified elsewhere.
Fixes#46069.
Beta-nominating because this is a regression for our new beta.
r? @nikomatsakis
Rustoc item summaries that are headers were not displayed at all because
they started with whitespace.
This PR fixes this and now removes the whitespace and then displays the
block.
We want librustdoc to pickup the env_logger dependency from
the sysroot. This ensures that the same copy of env_logger is used
for both internal crates (e.g. librustc_driver, libsyntax) and
librustdoc
Closes#46383
* Visibility was missing from impl items.
* Attributes and docs were missing from consts and types in impls.
* Const default values were missing from traits.
This unifies the code that handles associated items from impls and traits.
Reject '2' as a binary digit in internals of b: number formatting
The `radix!` macro generates an implementation of the private trait `GenericRadix`, and the code replaced changes Binary's implementation to no longer accept '2' as a valid digit to print.
Granted, this code is literally only ever called from another method in this private trait, and that method has logic to never hand a '2' to the printing function. Even given this, the code's there, I thought it would be best to fix this for clarity of anyone reading it.
impl From<bool> for AtomicBool
This seems like an obvious omission from #45610. ~~I've used the same feature name and version in the hope that this can be backported to beta so it's stabilized with the other impls. If it can't be I'll change it to `1.24.0`.~~
Improve documentation for slice swap/copy/clone operations.
Fixes#45636.
- Demonstrate how to use these operations with slices of differing
lengths
- Demonstrate how to swap/copy/clone sub-slices of a slice using
`split_at_mut`
Stabilize some `ascii_ctype` methods
As discussed in #39658, this PR stabilizes those methods for `u8` and `char`. All inherent `ascii_ctype` for `[u8]` and `str` are removed as we prefer the more explicit version `s.chars().all(|c| c.is_ascii_())`.
This PR doesn't modify the `AsciiExt` trait. There, the `ascii_ctype` methods are still unstable. It is planned to remove those in the future (I think). I had to modify some code in `ascii.rs` to properly implement `AsciiExt` for all types.
Fixes#39658.
Add std::sync::mpsc::Receiver::recv_deadline()
Essentially renames recv_max_until to recv_deadline (mostly copying recv_timeout
documentation). This function is useful to avoid the often unnecessary call to
Instant::now in recv_timeout (e.g. when the user already has a deadline). A
concrete example would be something along those lines:
```rust
use std::sync::mpsc::Receiver;
use std::time::{Duration, Instant};
/// Reads a batch of elements
///
/// Returns as soon as `max_size` elements have been received or `timeout` expires.
fn recv_batch_timeout<T>(receiver: &Receiver<T>, timeout: Duration, max_size: usize) -> Vec<T> {
recv_batch_deadline(receiver, Instant::now() + timeout, max_size)
}
/// Reads a batch of elements
///
/// Returns as soon as `max_size` elements have been received or `deadline` is reached.
fn recv_batch_deadline<T>(receiver: &Receiver<T>, deadline: Instant, max_size: usize) -> Vec<T> {
let mut result = Vec::new();
while let Ok(x) = receiver.recv_deadline(deadline) {
result.push(x);
if result.len() == max_size {
break;
}
}
result
}
```
