Optimize File::read_to_end and read_to_string
Reading a file into an empty vector or string buffer can incur unnecessary `read` syscalls and memory re-allocations as the buffer "warms up" and grows to its final size. This is perhaps a necessary evil with generic readers, but files can be read in smarter by checking the file size and reserving that much capacity.
`std::fs::read` and `std::fs::read_to_string` already perform this optimization: they open the file, reads its metadata, and call `with_capacity` with the file size. This ensures that the buffer does not need to be resized and an initial string of small `read` syscalls.
However, if a user opens the `File` themselves and calls `file.read_to_end` or `file.read_to_string` they do not get this optimization.
```rust
let mut buf = Vec::new();
file.read_to_end(&mut buf)?;
```
I searched through this project's codebase and even here are a *lot* of examples of this. They're found all over in unit tests, which isn't a big deal, but there are also several real instances in the compiler and in Cargo. I've documented the ones I found in a comment here:
https://github.com/rust-lang/rust/issues/89516#issuecomment-934423999
Most telling, the documentation for both the `Read` trait and the `Read::read_to_end` method both show this exact pattern as examples of how to use readers. What this says to me is that this shouldn't be solved by simply fixing the instances of it in this codebase. If it's here it's certain to be prevalent in the wider Rust ecosystem.
To that end, this commit adds specializations of `read_to_end` and `read_to_string` directly on `File`. This way it's no longer a minor footgun to start with an empty buffer when reading a file in.
A nice side effect of this change is that code that accesses a `File` as `impl Read` or `dyn Read` will benefit. For example, this code from `compiler/rustc_serialize/src/json.rs`:
```rust
pub fn from_reader(rdr: &mut dyn Read) -> Result<Json, BuilderError> {
let mut contents = Vec::new();
match rdr.read_to_end(&mut contents) {
```
Related changes:
- I also added specializations to `BufReader` to delegate to `self.inner`'s methods. That way it can call `File`'s optimized implementations if the inner reader is a file.
- The private `std::io::append_to_string` function is now marked `unsafe`.
- `File::read_to_string` being more efficient means that the performance note for `io::read_to_string` can be softened. I've added `@camelid's` suggested wording from https://github.com/rust-lang/rust/issues/80218#issuecomment-936806502.
r? `@joshtriplett`
Remove unwrap_or! macro
Removes `unwrap_or!` macro and replaces it with `match`.
It's kinda cleanup, as rustc_ast not the best place for this macro and this is used only in 2 places anyway.
Don't normalize xform_ret_ty during method candidate assembly
Fixes https://github.com/rust-lang/rust/issues/85671
Normalizing the return type of a method candidate together with the expected receiver type of the method can lead to valid method candidates being rejected during probing. Specifically in the example of the fixed issue we have a `self_ty` of the form `&A<&[Coef]>` whereas the `impl_ty` of the method would be `&A<_>`, if we normalize the projection in the return type we unify the inference variable with `Cont`, which will lead us to reject the candidate in the sup type check in `consider_probe`. Since we don't actually need the normalized return type during candidate assembly, we postpone the normalization until we consider candidates in `consider_probe`.
Default to disabling the new pass manager for the s390x arch targets.
This hack disables the new LLVM pass manager by default for s390x arch targets until the performance issues are fixed (see #89609). The command line option `-Z new-llvm-pass-manager=(yes|no)` continues to take precedence over this default.
Prevent error reporting from outputting a recursion error if it finds an ambiguous trait impl during suggestions
Closes#89275
This fixes the compiler reporting a recursion error during another already in progress error by trying to make a conversion method suggestion and encounters ambiguous trait implementations that can convert a the original type into a type that can then be recursively converted into itself via another method in the trait.
Updated OverflowError struct to be an enum so I could differentiate between passes - it's no longer a ZST but I don't think that should be a problem as they only generate when there's an error in compiling code anyway
Turn vtable_allocation() into a query
This PR removes the untracked vtable-const-allocation cache from the `tcx` and turns the `vtable_allocation()` method into a query.
The change is pretty straightforward and should be backportable without too much effort.
Fixes https://github.com/rust-lang/rust/issues/89598.
Correct decoding of foreign expansions during incr. comp.
Fixes https://github.com/rust-lang/rust/issues/74946
The original issue was due to a wrong assertion in `expn_hash_to_expn_id`.
The secondary issue was due to a mismatch between the encoding and decoding paths for expansions that are created after the TyCtxt is created.
Wrapper for `-Z gcc-ld=lld` to invoke rust-lld with the correct flavor
This PR adds an `lld-wrapper` tool which is installed as `ld` and `ld64` in `lib\rustlib\<host_target>\bin\gcc-ld` directory and whose sole purpose is to invoke `rust-lld` in the parent directory with the correct flavor. Lld decides which flavor to use from either the first two commandline arguments or from the name of the executable (`ld` for GNU/ld flavor, `ld64` for Darwin/Macos/ld64 flavor and so on). Symbolic links could not be used as they are not supported by rustup and on Windows.
The wrapper replaces full copies of rust-lld which added some significant bloat. On UNIXish operating systems it exec rust-lld, on Windows it spawns it as a child process.
Fixes#88869.
r? ```@Mark-Simulacrum```
cc ```@nagisa``` ```@petrochenkov``` ```@1000teslas```
Implement #85440 (Random test ordering)
This PR adds `--shuffle` and `--shuffle-seed` options to `libtest`. The options are similar to the [`-shuffle` option](c894b442d1/src/testing/testing.go (L1482-L1499)) that was recently added to Go.
Here are the relevant parts of the help message:
```
--shuffle Run tests in random order
--shuffle-seed SEED
Run tests in random order; seed the random number
generator with SEED
...
By default, the tests are run in alphabetical order. Use --shuffle or set
RUST_TEST_SHUFFLE to run the tests in random order. Pass the generated
"shuffle seed" to --shuffle-seed (or set RUST_TEST_SHUFFLE_SEED) to run the
tests in the same order again. Note that --shuffle and --shuffle-seed do not
affect whether the tests are run in parallel.
```
Is an RFC needed for this?
Implement `#[link_ordinal(n)]`
Allows the use of `#[link_ordinal(n)]` with `#[link(kind = "raw-dylib")]`, allowing Rust to link against DLLs that export symbols by ordinal rather than by name. As long as the ordinal matches, the name of the function in Rust is not required to match the name of the corresponding function in the exporting DLL.
Part of #58713.
On macOS, make strip="symbols" not pass any options to strip
This makes the output with `strip="symbols"` match the result of just
calling `strip` on the output binary, minimizing the size of the binary.
Enable AutoFDO.
This largely involves implementing the options debug-info-for-profiling
and profile-sample-use and forwarding them on to LLVM.
AutoFDO can be used on x86-64 Linux like this:
rustc -O -Clink-arg='Wl,--no-rosegment' -Cdebug-info-for-profiling main.rs -o main
perf record -b ./main
create_llvm_prof --binary=main --out=code.prof
rustc -O -Cprofile-sample-use=code.prof main.rs -o main2
Now `main2` will have feedback directed optimization applied to it.
The create_llvm_prof tool can be obtained from this github repository:
https://github.com/google/autofdo
The option -Clink-arg='Wl,--no-rosegment' is necessary to avoid lld
putting an extra RO segment before the executable code, which would make
the binary silently incompatible with create_llvm_prof.
Reading a file into an empty vector or string buffer can incur
unnecessary `read` syscalls and memory re-allocations as the buffer
"warms up" and grows to its final size. This is perhaps a necessary evil
with generic readers, but files can be read in smarter by checking the
file size and reserving that much capacity.
`std::fs::read` and `read_to_string` already perform this optimization:
they open the file, reads its metadata, and call `with_capacity` with
the file size. This ensures that the buffer does not need to be resized
and an initial string of small `read` syscalls.
However, if a user opens the `File` themselves and calls
`file.read_to_end` or `file.read_to_string` they do not get this
optimization.
```rust
let mut buf = Vec::new();
file.read_to_end(&mut buf)?;
```
I searched through this project's codebase and even here are a *lot* of
examples of this. They're found all over in unit tests, which isn't a
big deal, but there are also several real instances in the compiler and
in Cargo. I've documented the ones I found in a comment here:
https://github.com/rust-lang/rust/issues/89516#issuecomment-934423999
Most telling, the `Read` trait and the `read_to_end` method both show
this exact pattern as examples of how to use readers. What this says to
me is that this shouldn't be solved by simply fixing the instances of it
in this codebase. If it's here it's certain to be prevalent in the wider
Rust ecosystem.
To that end, this commit adds specializations of `read_to_end` and
`read_to_string` directly on `File`. This way it's no longer a minor
footgun to start with an empty buffer when reading a file in.
A nice side effect of this change is that code that accesses a `File` as
a bare `Read` constraint or via a `dyn Read` trait object will benefit.
For example, this code from `compiler/rustc_serialize/src/json.rs`:
```rust
pub fn from_reader(rdr: &mut dyn Read) -> Result<Json, BuilderError> {
let mut contents = Vec::new();
match rdr.read_to_end(&mut contents) {
```
Related changes:
- I also added specializations to `BufReader` to delegate to
`self.inner`'s methods. That way it can call `File`'s optimized
implementations if the inner reader is a file.
- The private `std::io::append_to_string` function is now marked
`unsafe`.
- `File::read_to_string` being more efficient means that the performance
note for `io::read_to_string` can be softened. I've added @camelid's
suggested wording from:
https://github.com/rust-lang/rust/issues/80218#issuecomment-936806502
Rollup of 7 pull requests
Successful merges:
- #89298 (Issue 89193 - Fix ICE when using `usize` and `isize` with SIMD gathers )
- #89461 (Add `deref_into_dyn_supertrait` lint.)
- #89477 (Move items related to computing diffs to a separate file)
- #89559 (RustWrapper: adapt for LLVM API change)
- #89585 (Emit item no type error even if type inference fails)
- #89596 (Make cfg imply doc(cfg))
- #89615 (Add InferCtxt::with_opaque_type_inference to get_body_with_borrowck_facts)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
The wrapper is installed as `ld` and `ld64` in the `lib\rustlib\<host_target>\bin\gcc-ld`
directory and its sole purpose is to invoke `rust-lld` in the parent directory with
the correct flavor.
Make cfg imply doc(cfg)
This is a reopening of #79341, rebased and modified a bit (we made a lot of refactoring in rustdoc's types so they needed to be reflected in this PR as well):
* `hidden_cfg` is now in the `Cache` instead of `DocContext` because `cfg` information isn't stored anymore on `clean::Attributes` type but instead computed on-demand, so we need this information in later parts of rustdoc.
* I removed the `bool_to_options` feature (which makes the code a bit simpler to read for `SingleExt` trait implementation.
* I updated the version for the feature.
There is only one thing I couldn't figure out: [this comment](https://github.com/rust-lang/rust/pull/79341#discussion_r561855624)
> I think I'll likely scrap the whole `SingleExt` extension trait as the diagnostics for 0 and >1 items should be different.
How/why should they differ?
EDIT: this part has been solved, the current code was fine, just needed a little simplification.
cc `@Nemo157`
r? `@jyn514`
Original PR description:
This is only active when the `doc_cfg` feature is active.
The implicit cfg can be overridden via `#[doc(cfg(...))]`, so e.g. to hide a `#[cfg]` you can use something like:
```rust
#[cfg(unix)]
#[doc(cfg(all()))]
pub struct Unix;
```
By adding `#![doc(cfg_hide(foobar))]` to the crate attributes the cfg `#[cfg(foobar)]` (and _only_ that _exact_ cfg) will not be implicitly treated as a `doc(cfg)` to render a message in the documentation.
RustWrapper: adapt for LLVM API change
No functional changes intended.
The LLVM commit
e463b69736
changed an argument of fatal_error_handler_t from std::string to char*.
This adapts RustWrapper accordingly.
Add `deref_into_dyn_supertrait` lint.
Initial implementation of #89460. Resolves#89190.
Maybe also worth a beta backport if necessary.
r? `@nikomatsakis`
Introduce `tcx.get_diagnostic_name`
Introduces a "reverse lookup" for diagnostic items. This is mainly intended for `@rust-lang/clippy` which often does a long series of `is_diagnostic_item` calls for the same `DefId`.
r? `@oli-obk`
