Remove `#[cfg(miri)]` from OnceCell tests
They were carried over from once_cell crate, but they are not entirely
correct (as miri now supports more things), and we don't run miri
tests for std, so let's just remove them.
Maybe one day we'll run miri in std, but then we can just re-install
these attributes.
Move to intra doc links for std::io
Helps with #75080.
@rustbot modify labels: T-doc, A-intra-doc-links, T-rustdoc
r? @jyn514
I had no problems with those files so I added some small links here and there.
They were carried over from once_cell crate, but they are not entirely
correct (as miri now supports more things), and we don't run miri
tests for std, so let's just remove them.
Maybe one day we'll run miri in std, but then we can just re-install
these attributes.
Switch to intra-doc links in /src/sys/unix/ext/*.rs
Partial fix for #75080
@rustbot modify labels: T-doc, A-intra-doc-links, T-rustdoc
r? @jyn514
These two links are not resolving to either `crate::fs::File...` or `fs::File...`
```
# unix/ext/fs.rs
27: /// [`File::read`]: ../../../../std/fs/struct.File.html#method.read
130: /// [`File::write`]: ../../../../std/fs/struct.File.html#method.write
```
Move to intra doc links for ascii.rs and panic.rs
Helps with #75080.
@rustbot modify labels: T-doc, A-intra-doc-links, T-rustdoc
I also updated the doc to fix the wording in `AsciiExt` since it is now deprecated.
The two file are small changes so I bundled them together.
Some links could not be changed to make them work, I believe those are known issues with primitive types.
Move to intra doc links in std::net
Helps with #75080.
@rustbot modify labels: T-doc, A-intra-doc-links, T-rustdoc
The links for `true` and `false` had to stay else `rustdoc` complained, it is intended ?
Add sanitizer support on FreeBSD
Restarting #47337. Everything is better now, no more weird llvm problems, well not everything:
Unfortunately, the sanitizers don't have proper support for versioned symbols (https://github.com/google/sanitizers/issues/628), so `libc`'s usage of `stat@FBSD_1.0` and so on explodes, e.g. in calling `std::fs::metadata`.
Building std (now easy thanks to cargo `-Zbuild-std`) and libc with `freebsd12/13` config via the `LIBC_CI=1` env variable is a good workaround…
```
LIBC_CI=1 RUSTFLAGS="-Z sanitizer=address" cargo +san-test -Zbuild-std run --target x86_64-unknown-freebsd --verbose
```
…*except* std won't build because there's no `st_lspare` in the ino64 version of the struct, so an std patch is required:
```diff
--- i/src/libstd/os/freebsd/fs.rs
+++ w/src/libstd/os/freebsd/fs.rs
@@ -66,8 +66,6 @@ pub trait MetadataExt {
fn st_flags(&self) -> u32;
#[stable(feature = "metadata_ext2", since = "1.8.0")]
fn st_gen(&self) -> u32;
- #[stable(feature = "metadata_ext2", since = "1.8.0")]
- fn st_lspare(&self) -> u32;
}
#[stable(feature = "metadata_ext", since = "1.1.0")]
@@ -136,7 +134,4 @@ impl MetadataExt for Metadata {
fn st_flags(&self) -> u32 {
self.as_inner().as_inner().st_flags as u32
}
- fn st_lspare(&self) -> u32 {
- self.as_inner().as_inner().st_lspare as u32
- }
}
```
I guess std could like.. detect that `libc` isn't built for the old ABI, and replace the implementation of `st_lspare` with a panic?
std/sys/unix/time: make it easier for LLVM to optimize `Instant` subtraction.
This PR is the minimal change necessary to get LLVM to optimize `if self.t.tv_nsec >= other.t.tv_nsec` to branchless instructions (at least on x86_64), inspired by @m-ou-se's own attempts at optimizing `Instant` subtraction.
I stumbled over this by looking at the total number of instructions executed by `rustc -Z self-profile`, and found that after disabling ASLR, the largest source of non-determinism remaining was from this `if` taking one branch or the other, depending on the values involved.
The reason this code is even called so many times to make a difference, is that `measureme` (the `-Z self-profile` implementation) currently uses `Instant::elapsed` for its event timestamps (of which there can be millions).
I doubt it's critical to land this, although perhaps it could slightly improve some forms of benchmarking.
Change Debug impl of SocketAddr and IpAddr to match their Display output
This has already been done for `SocketAddrV4`, `SocketAddrV6`, `IpAddrV4` and `IpAddrV6`. I don't see a point to keep the rather bad to read derived impl, especially so when pretty printing:
V4(
127.0.0.1
)
From the `Display`, one can easily and unambiguously see if it's V4 or V6. Two examples:
```
127.0.0.1:443
[2001:db8:85a3::8a2e:370:7334]:443
```
Luckily the docs explicitly state that `Debug` output is not stable and that it may be changed at any time.
Using `Display` as `Debug` is very convenient for configuration structs (e.g. for webservers) that often just have a `derive(Debug)` and are printed that way to the one starting the server.
Improve documentation on process::Child.std* fields
As a relative beginner, it took a while for me to figure out I could just steal the references to avoid partially moving the child and thus retain ability to call functions on it (and store it in structs etc).
This solves several problems
- race conditions where a file is truncated while copying from it. if we blindly trusted
the file size this would lead to an infinite loop
- proc files appearing empty to copy_file_range but not to read/write
https://github.com/coreutils/coreutils/commit/4b04a0c
- copy_file_range returning 0 for some filesystems (overlay? bind mounts?)
inside docker, again leading to an infinite loop
As a relative beginner, it took a while for me to figure out I could just steal the references to avoid partially moving the child and thus retain ability to call functions on it (and store it in structs etc).
Expand function pointer docs
Be more explicit in the ABI section, and add a section on how to obtain a function pointer, which can be somewhat confusing.
Cc https://github.com/rust-lang/rust/issues/75239
Move to intra doc links whenever possible within std/src/lib.rs
Helps with #75080.
@rustbot modify labels: T-doc, A-intra-doc-links, T-rustdoc
There are some things like
```rust
`//! [`Option<T>`]: option::Option`
```
that will either be fixed in the future or have open issues about them.
Fix minor things in the `f32` primitive docs
All of these were review comments in #74621 that I first fixed in that PR, but later accidentally overwrote by a force push.
Thanks @the8472 for noticing.
r? @KodrAus
Fix wasi::fs::OpenOptions to imply write when append is on
This PR fixes a bug in `OpenOptions` of `wasi` platform that it currently doesn't imply write mode when only `append` is enabled.
As explained in the [doc of OpenOptions#append](https://doc.rust-lang.org/std/fs/struct.OpenOptions.html#method.append), calling `.append(true)` should imply `.write(true)` as well.
## Reproduce
Given below simple Rust program:
```rust
use std::fs::OpenOptions;
use std::io::Write;
fn main() {
let mut file = OpenOptions::new()
.write(true)
.create(true)
.open("foo.txt")
.unwrap();
writeln!(file, "abc").unwrap();
}
```
it can successfully compiled into wasm and execute by `wasmtime` runtime:
```sh
$ rustc --target wasm32-wasi write.rs
$ ~/wasmtime/target/debug/wasmtime run --dir=. write.wasm
$ cat foo.txt
abc
```
However when I change `.write(true)` to `.append(true)`, it fails to execute by the error "Capabilities insufficient":
```sh
$ ~/wasmtime/target/debug/wasmtime run --dir=. append.wasm
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 76, kind: Other, message: "Capabilities insufficient" }', append.rs:10:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Error: failed to run main module `append.wasm`
...
```
This is because of lacking "rights" on the opened file:
```sh
$ RUST_LOG=trace ~/wasmtime/target/debug/wasmtime run --dir=. append.wasm 2>&1 | grep validate_rights
TRACE wasi_common::entry > | validate_rights failed: required rights = HandleRights { base: fd_write (0x40), inheriting: empty (0x0) }; actual rights = HandleRights { base: fd_seek|fd_fdstat_set_flags|fd_sync|fd_tell|fd_advise|fd_filestat_set_times|poll_fd_readwrite (0x88000bc), inheriting: empty (0x0) }
```
Add Ipv6Addr::to_ipv4_mapped
* add Ipv6Addr::to_ipv4_mapped
* ~~deprecate Ipv4Addr::to_ipv6_compatible & Ipv6Addr::to_ipv4~~ reference: #75150
According to [IETF RFC 4291](https://tools.ietf.org/html/rfc4291#page-10), the "IPv4-Compatible IPv6 address" is deprecated.
> 2.5.5.1. IPv4-Compatible IPv6 Address
>
> The "IPv4-Compatible IPv6 address" was defined to assist in the IPv6
> transition. The format of the "IPv4-Compatible IPv6 address" is as
> follows:
>
> | 80 bits | 16 | 32 bits |
> +--------------------------------------+--------------------------+
> |0000..............................0000|0000| IPv4 address |
> +--------------------------------------+----+---------------------+
>
> Note: The IPv4 address used in the "IPv4-Compatible IPv6 address"
> must be a globally-unique IPv4 unicast address.
>
> The "IPv4-Compatible IPv6 address" is now deprecated because the
> current IPv6 transition mechanisms no longer use these addresses.
> New or updated implementations are not required to support this
> address type.
And the current implementation of `Ipv4Addr::to_ipv6_compatible`is incorrect: it does not check whether the IPv4 address is a globally-unique IPv4 unicast address.
Please let me know if there are any issues with this pull request.
Improve `f32` and `f64` primitive documentation
I noticed that the docs for the primitive floats were fairly short. I first only wanted to add the IEEE specification information (compare [the reference](https://doc.rust-lang.org/reference/types/numeric.html)), but then also added some more beginner-friendly docs. Let me know what you think!
Random doc team assign:
r? @rylev
Std panicking unsafe block in unsafe fn
Partial fix of #73904.
This encloses `unsafe` operations in `unsafe fn` in `libstd/ffi/panicking.rs`.
I also made a two lines change to `libstd/thread/local.rs` to add the necessary `unsafe` block without breaking everything else.
@rustbot modify labels: F-unsafe-block-in-unsafe-fn
Unfortunately, sanitizers do not support versioned symbols[1],
so they break filesystem access via the legacy, pre-ino64 ABI.
To use sanitizers on FreeBSD >= 12, we need to build the libc
crate with LIBC_CI=1 to use the new ABI -- including the libc
used for std. But that removes the st_lspare field std was
expecting for the deprecated metadata extension.
Add a way to skip that field to allow the build to work.
[1]: https://github.com/google/sanitizers/issues/628
Move to intra-doc links in library/std/src/path.rs
Helps with #75080.
@rustbot modify labels: T-doc, A-intra-doc-links, T-rustdoc
Known issue: The following links are broken (they are inside trait impls, undocumented in this file, inheriting from the original doc):
- [`Hasher`]
- [`Self`] (referencing `../primitive.slice.html`)
- [`Ordering`]
remove_dir will error if the path doesn't exist or isn't a directory.
It's useful to clarify that this is "remove dir or fail" not "remove dir
if it exists".
I don't think this belongs in the title. "Removes an existing, empty
directory" is strangely worded-- there's no such thing as a non-existing
directory. Better to just say explicitly it will return an error.
Implement `into_keys` and `into_values` for associative maps
This PR implements `into_keys` and `into_values` for HashMap and BTreeMap types. They are implemented as unstable, under `map_into_keys_values` feature.
Fixes#55214.
r? @dtolnay
All #[cfg(unix)] platforms follow the POSIX standard and define _SC_IOV_MAX so
that we rely purely on POSIX semantics to determine the limits on I/O vector
count.
Keep the I/O vector count limit in a `SyncOnceCell` to avoid the overhead of
repeatedly calling `sysconf` as these limits are guaranteed to not change during
the lifetime of a process by POSIX.
Both Linux and MacOS enforce limits on the vector count when performing vectored
I/O via the readv and writev system calls and return EINVAL when these limits
are exceeded. This changes the standard library to handle those limits as short
reads and writes to avoid forcing its users to query these limits using
platform specific mechanisms.
Co-authored-by: Weiyi Wang <wwylele@gmail.com>
Co-authored-by: Adam Reichold <adam.reichold@t-online.de>
Co-authored-by: Josh Stone <cuviper@gmail.com>
Co-authored-by: Scott McMurray <scottmcm@users.noreply.github.com>
Co-authored-by: tmiasko <tomasz.miasko@gmail.com>
Previously `std::fs::File::metadata` on wasm32-wasi would call `fd_filestat_get`
to get metadata associated with fd, but that fd is opened without
RIGHTS_FD_FILESTAT_GET right, so it will failed on correctly implemented WASI
environment.
This change instead to add the missing rights when opening an fd.
Remove links to rejected errata 4406 for RFC 4291
Fixes#74198.
For now I simply removed the links, the docs seems clear enough to me but I'm no expert in the domain so don't hesitate to correct me if more is needed.
cc @ghanan94.
@rustbot modify labels: T-doc, T-libs
Fix RefUnwindSafe & UnwinsSafe impls for lazy::SyncLazy
I *think* we should implement those unconditionally with respect to `F`.
The user code can't observe the closure in any way, and we poison lazy if the closure itself panics.
But I've never fully wrapped my head around `UnwindSafe` traits, so 🤷♂️
This commit is a proof-of-concept for switching the standard library's
backtrace symbolication mechanism on most platforms from libbacktrace to
gimli. The standard library's support for `RUST_BACKTRACE=1` requires
in-process parsing of object files and DWARF debug information to
interpret it and print the filename/line number of stack frames as part
of a backtrace.
Historically this support in the standard library has come from a
library called "libbacktrace". The libbacktrace library seems to have
been extracted from gcc at some point and is written in C. We've had a
lot of issues with libbacktrace over time, unfortunately, though. The
library does not appear to be actively maintained since we've had
patches sit for months-to-years without comments. We have discovered a
good number of soundness issues with the library itself, both when
parsing valid DWARF as well as invalid DWARF. This is enough of an issue
that the libs team has previously decided that we cannot feed untrusted
inputs to libbacktrace. This also doesn't take into account the
portability of libbacktrace which has been difficult to manage and
maintain over time. While possible there are lots of exceptions and it's
the main C dependency of the standard library right now.
For years it's been the desire to switch over to a Rust-based solution
for symbolicating backtraces. It's been assumed that we'll be using the
Gimli family of crates for this purpose, which are targeted at safely
and efficiently parsing DWARF debug information. I've been working
recently to shore up the Gimli support in the `backtrace` crate. As of a
few weeks ago the `backtrace` crate, by default, uses Gimli when loaded
from crates.io. This transition has gone well enough that I figured it
was time to start talking seriously about this change to the standard
library.
This commit is a preview of what's probably the best way to integrate
the `backtrace` crate into the standard library with the Gimli feature
turned on. While today it's used as a crates.io dependency, this commit
switches the `backtrace` crate to a submodule of this repository which
will need to be updated manually. This is not done lightly, but is
thought to be the best solution. The primary reason for this is that the
`backtrace` crate needs to do some pretty nontrivial filesystem
interactions to locate debug information. Working without `std::fs` is
not an option, and while it might be possible to do some sort of
trait-based solution when prototyped it was found to be too unergonomic.
Using a submodule allows the `backtrace` crate to build as a submodule
of the `std` crate itself, enabling it to use `std::fs` and such.
Otherwise this adds new dependencies to the standard library. This step
requires extra attention because this means that these crates are now
going to be included with all Rust programs by default. It's important
to note, however, that we're already shipping libbacktrace with all Rust
programs by default and it has a bunch of C code implementing all of
this internally anyway, so we're basically already switching
already-shipping functionality to Rust from C.
* `object` - this crate is used to parse object file headers and
contents. Very low-level support is used from this crate and almost
all of it is disabled. Largely we're just using struct definitions as
well as convenience methods internally to read bytes and such.
* `addr2line` - this is the main meat of the implementation for
symbolication. This crate depends on `gimli` for DWARF parsing and
then provides interfaces needed by the `backtrace` crate to turn an
address into a filename / line number. This crate is actually pretty
small (fits in a single file almost!) and mirrors most of what
`dwarf.c` does for libbacktrace.
* `miniz_oxide` - the libbacktrace crate transparently handles
compressed debug information which is compressed with zlib. This crate
is used to decompress compressed debug sections.
* `gimli` - not actually used directly, but a dependency of `addr2line`.
* `adler32`- not used directly either, but a dependency of
`miniz_oxide`.
The goal of this change is to improve the safety of backtrace
symbolication in the standard library, especially in the face of
possibly malformed DWARF debug information. Even to this day we're still
seeing segfaults in libbacktrace which could possibly become security
vulnerabilities. This change should almost entirely eliminate this
possibility whilc also paving the way forward to adding more features
like split debug information.
Some references for those interested are:
* Original addition of libbacktrace - #12602
* OOM with libbacktrace - #24231
* Backtrace failure due to use of uninitialized value - #28447
* Possibility to feed untrusted data to libbacktrace - #21889
* Soundness fix for libbacktrace - #33729
* Crash in libbacktrace - #39468
* Support for macOS, never merged - ianlancetaylor/libbacktrace#2
* Performance issues with libbacktrace - #29293, #37477
* Update procedure is quite complicated due to how many patches we
need to carry - #50955
* Libbacktrace doesn't work on MinGW with dynamic libs - #71060
* Segfault in libbacktrace on macOS - #71397
Switching to Rust will not make us immune to all of these issues. The
crashes are expected to go away, but correctness and performance may
still have bugs arise. The gimli and `backtrace` crates, however, are
actively maintained unlike libbacktrace, so this should enable us to at
least efficiently apply fixes as situations come up.
This commit updates the src/stdarch submodule primarily to include
rust-lang/stdarch#874 which updated and revamped WebAssembly SIMD
intrinsics and renamed WebAssembly atomics intrinsics. This is all
unstable surface area of the standard library so the changes should be
ok here. The SIMD updates also enable SIMD intrinsics to be used by any
program any any time, yay!
cc #74372, a tracking issue I've opened for the stabilization of SIMD
intrinsics
This has already been done for `SocketAddrV4`, `SocketAddrV6`,
`IpAddrV4` and `IpAddrV6`. I don't see a point to keep the rather bad
to read derived impl, especially when pretty printing:
V4(
127.0.0.1
)
From the `Display`, one can easily and unambiguously see if it's V4 or
V6. Using `Display` as `Debug` is very convenient for configuration
structs (e.g. for webservers) that often just have a `derive(Debug)`
and are printed that way to the user.
