Issue #84096 changed the hashmap RNG to use BCryptGenRandom instead of
RtlGenRandom on Windows.
Mozilla Firefox started experiencing random failures in
env_logger::Builder::new() (Issue #94098) during initialization of their
unsandboxed main process with an "Access Denied" error message from
BCryptGenRandom(), which is used by the HashMap contained in
env_logger::Builder
The root cause appears to be a virus scanner or other software interfering
with BCrypt DLLs loading.
This change adds a fallback option if BCryptGenRandom is unusable for
whatever reason. It will fallback to RtlGenRandom in this case.
Fixes#94098
Add a dedicated length-prefixing method to `Hasher`
This accomplishes two main goals:
- Make it clear who is responsible for prefix-freedom, including how they should do it
- Make it feasible for a `Hasher` that *doesn't* care about Hash-DoS resistance to get better performance by not hashing lengths
This does not change rustc-hash, since that's in an external crate, but that could potentially use it in future.
Fixes#94026
r? rust-lang/libs
---
The core of this change is the following two new methods on `Hasher`:
```rust
pub trait Hasher {
/// Writes a length prefix into this hasher, as part of being prefix-free.
///
/// If you're implementing [`Hash`] for a custom collection, call this before
/// writing its contents to this `Hasher`. That way
/// `(collection![1, 2, 3], collection![4, 5])` and
/// `(collection![1, 2], collection![3, 4, 5])` will provide different
/// sequences of values to the `Hasher`
///
/// The `impl<T> Hash for [T]` includes a call to this method, so if you're
/// hashing a slice (or array or vector) via its `Hash::hash` method,
/// you should **not** call this yourself.
///
/// This method is only for providing domain separation. If you want to
/// hash a `usize` that represents part of the *data*, then it's important
/// that you pass it to [`Hasher::write_usize`] instead of to this method.
///
/// # Examples
///
/// ```
/// #![feature(hasher_prefixfree_extras)]
/// # // Stubs to make the `impl` below pass the compiler
/// # struct MyCollection<T>(Option<T>);
/// # impl<T> MyCollection<T> {
/// # fn len(&self) -> usize { todo!() }
/// # }
/// # impl<'a, T> IntoIterator for &'a MyCollection<T> {
/// # type Item = T;
/// # type IntoIter = std::iter::Empty<T>;
/// # fn into_iter(self) -> Self::IntoIter { todo!() }
/// # }
///
/// use std:#️⃣:{Hash, Hasher};
/// impl<T: Hash> Hash for MyCollection<T> {
/// fn hash<H: Hasher>(&self, state: &mut H) {
/// state.write_length_prefix(self.len());
/// for elt in self {
/// elt.hash(state);
/// }
/// }
/// }
/// ```
///
/// # Note to Implementers
///
/// If you've decided that your `Hasher` is willing to be susceptible to
/// Hash-DoS attacks, then you might consider skipping hashing some or all
/// of the `len` provided in the name of increased performance.
#[inline]
#[unstable(feature = "hasher_prefixfree_extras", issue = "88888888")]
fn write_length_prefix(&mut self, len: usize) {
self.write_usize(len);
}
/// Writes a single `str` into this hasher.
///
/// If you're implementing [`Hash`], you generally do not need to call this,
/// as the `impl Hash for str` does, so you can just use that.
///
/// This includes the domain separator for prefix-freedom, so you should
/// **not** call `Self::write_length_prefix` before calling this.
///
/// # Note to Implementers
///
/// The default implementation of this method includes a call to
/// [`Self::write_length_prefix`], so if your implementation of `Hasher`
/// doesn't care about prefix-freedom and you've thus overridden
/// that method to do nothing, there's no need to override this one.
///
/// This method is available to be overridden separately from the others
/// as `str` being UTF-8 means that it never contains `0xFF` bytes, which
/// can be used to provide prefix-freedom cheaper than hashing a length.
///
/// For example, if your `Hasher` works byte-by-byte (perhaps by accumulating
/// them into a buffer), then you can hash the bytes of the `str` followed
/// by a single `0xFF` byte.
///
/// If your `Hasher` works in chunks, you can also do this by being careful
/// about how you pad partial chunks. If the chunks are padded with `0x00`
/// bytes then just hashing an extra `0xFF` byte doesn't necessarily
/// provide prefix-freedom, as `"ab"` and `"ab\u{0}"` would likely hash
/// the same sequence of chunks. But if you pad with `0xFF` bytes instead,
/// ensuring at least one padding byte, then it can often provide
/// prefix-freedom cheaper than hashing the length would.
#[inline]
#[unstable(feature = "hasher_prefixfree_extras", issue = "88888888")]
fn write_str(&mut self, s: &str) {
self.write_length_prefix(s.len());
self.write(s.as_bytes());
}
}
```
With updates to the `Hash` implementations for slices and containers to call `write_length_prefix` instead of `write_usize`.
`write_str` defaults to using `write_length_prefix` since, as was pointed out in the issue, the `write_u8(0xFF)` approach is insufficient for hashers that work in chunks, as those would hash `"a\u{0}"` and `"a"` to the same thing. But since `SipHash` works byte-wise (there's an internal buffer to accumulate bytes until a full chunk is available) it overrides `write_str` to continue to use the add-non-UTF-8-byte approach.
---
Compatibility:
Because the default implementation of `write_length_prefix` calls `write_usize`, the changed hash implementation for slices will do the same thing the old one did on existing `Hasher`s.
Use futex-based locks and thread parker on {Free, Open, DragonFly}BSD.
This switches *BSD to our futex-based locks and thread parker.
Tracking issue: https://github.com/rust-lang/rust/issues/93740
This is a draft, because this still needs a new version of the `libc` crate to be published that includes https://github.com/rust-lang/libc/pull/2770.
r? `@Amanieu`
This accomplishes two main goals:
- Make it clear who is responsible for prefix-freedom, including how they should do it
- Make it feasible for a `Hasher` that *doesn't* care about Hash-DoS resistance to get better performance by not hashing lengths
This does not change rustc-hash, since that's in an external crate, but that could potentially use it in future.
Implement [OsStr]::join
Implements join for `OsStr` and `OsString` slices:
```Rust
let strings = [OsStr::new("hello"), OsStr::new("dear"), OsStr::new("world")];
assert_eq!("hello dear world", strings.join(OsStr::new(" ")));
````
This saves one from converting to strings and back, or from implementing it manually.
Relax memory ordering used in SameMutexCheck
`SameMutexCheck` only requires atomicity for `self.addr`, but does not need ordering of other memory accesses in either the success or failure case. Using `Relaxed`, the code still correctly handles the case when two threads race to store an address.
Relax memory ordering used in `min_stack`
`min_stack` does not provide any synchronization guarantees to its callers, and only requires atomicity for `MIN` itself, so relaxed memory ordering is sufficient.
`SameMutexCheck` only requires atomicity for `self.addr`, but does not need ordering of other memory accesses in either the success or failure case. Using `Relaxed`, the code still correctly handles the case when two threads race to store an address.
`min_stack` does not provide any synchronization guarantees to its callers, and only requires atomicity for `MIN` itself, so relaxed memory ordering is sufficient.
rustdoc: Resolve doc links referring to `macro_rules` items
cc https://github.com/rust-lang/rust/issues/81633
UPD: the fallback to considering *all* `macro_rules` in the crate for unresolved names is not removed in this PR, it will be removed separately and will be run through crater.
Make [e]println macros eagerly drop temporaries (for backport)
This PR extracts the subset of #96455 which is only the parts necessary for fixing the 1.61-beta regressions in #96434.
My larger PR #96455 contains a few other changes relative to the pre-#94868 behavior; those are not necessary to backport into 1.61.
argument position | before #94868 | after #94868 | after this PR
--- |:---:|:---:|:---:
`write!($tmp, "…", …)` | 😡 | 😡 | 😡
`write!(…, "…", $tmp)` | 😡 | 😡 | 😡
`writeln!($tmp, "…", …)` | 😡 | 😡 | 😡
`writeln!(…, "…", $tmp)` | 😡 | 😡 | 😡
`print!("…", $tmp)` | 😡 | 😡 | 😡
`println!("…", $tmp)` | 😺 | 😡 | 😺
`eprint!("…", $tmp)` | 😡 | 😡 | 😡
`eprintln!("…", $tmp)` | 😺 | 😡 | 😺
`panic!("…", $tmp)` | 😺 | 😺 | 😺
Revert "Re-export core::ffi types from std::ffi"
This reverts commit 9aed829fe6.
Fixes https://github.com/rust-lang/rust/issues/96435 , a regression
in crates doing `use std::ffi::*;` and `use std::os::raw::*;`.
We can re-add this re-export once the `core::ffi` types
are stable, and thus the `std::os::raw` types can become re-exports as
well, which will avoid the conflict. (Type aliases to the same type
still conflict, but re-exports of the same type don't.)
Windows: Make stdin pipes synchronous
Stdin pipes do not need to be used asynchronously within the standard library. This is a first step in making pipes mostly synchronous.
r? `@m-ou-se`
std: directly use pthread in UNIX parker implementation
`Mutex` and `Condvar` are being replaced by more efficient implementations, which need thread parking themselves (see #93740). Therefore we should use the `pthread` synchronization primitives directly. Also, we can avoid allocating the mutex and condition variable because the `Parker` struct is being placed in an `Arc` anyways.
This basically is just a copy of the current `Mutex` and `Condvar` code, which will however be removed (again, see #93740). An alternative implementation could be to use dedicated private `OsMutex` and `OsCondvar` types, but all the other platforms supported by std actually have their own thread parking primitives.
I used `Pin` to guarantee a stable address for the `Parker` struct, while the current implementation does not, rather using extra unsafe declaration. Since the thread struct is shared anyways, I assumed this would not add too much clutter while being clearer.
Make EncodeWide implement FusedIterator
[`EncodeUtf16`](https://doc.rust-lang.org/std/str/struct.EncodeUtf16.html) and [`EncodeWide`](https://doc.rust-lang.org/std/os/windows/ffi/struct.EncodeWide.html) currently serve similar purposes: They convert from UTF-8 to UTF-16 and WTF-8 to WTF-16, respectively. `EncodeUtf16` wraps a &str, whereas `EncodeWide` wraps an &OsStr.
When Iteration has concluded, these iterators wrap an empty slice, which will forever yield `None` values. Hence, `EncodeUtf16` rightfully implements `FusedIterator`. However, `EncodeWide` in contrast does not, even though it serves an almost identical purpose.
This PR attempts to fix that issue. I consider this change minor and non-controversial, hence why I have not added a RFC/FCP. Please let me know if the stability attribute is wrong or contains a wrong version number. Thanks in advance.
Fixes https://github.com/rust-lang/rust/issues/96368
This reverts commit 9aed829fe6.
Fixes https://github.com/rust-lang/rust/issues/96435 , a regression
in crates doing `use std::ffi::*;` and `use std::os::raw::*;`.
We can re-add this re-export once the `core::ffi` types
are stable, and thus the `std::os::raw` types can become re-exports as
well, which will avoid the conflict. (Type aliases to the same type
still conflict, but re-exports of the same type don't.)
Define a dedicated error type for `HandleOrNull` and `HandleOrInvalid`.
Define `NullHandleError` and `InvalidHandleError` types, that implement std::error::Error, and use them as the error types in `HandleOrNull` and `HandleOrInvalid`,
This addresses [this concern](https://github.com/rust-lang/rust/issues/87074#issuecomment-1080031167).
This is the same as #95387.
r? `@joshtriplett`
Mutex and Condvar are being replaced by more efficient implementations, which need thread parking themselves (see #93740). Therefore use the pthread synchronization primitives directly. Also, avoid allocating because the Parker struct is being placed in an Arc anyways.
Windows Command: Don't run batch files using verbatim paths
Fixes#95178
Note that the first commit does some minor refactoring (moving command line argument building to args.rs). The actual changes are in the second.
Reduce allocations for path conversions on Windows
Previously, UTF-8 to UTF-16 Path conversions on Windows unnecessarily allocate twice, as described in #96297. This commit fixes that issue.
Improve Windows path prefix parsing
This PR fixes improves parsing of Windows path prefixes. `parse_prefix` now supports both types of separators on Windows (`/` and `\`).
