Allow writing of incomplete UTF-8 sequences to the Windows console via stdout/stderr
# Problem
Writes of just an incomplete UTF-8 byte sequence (e.g. `b"\xC3"` or `b"\xF0\x9F"`) to stdout/stderr with a Windows console attached error with `io::ErrorKind::InvalidData, "Windows stdio in console mode does not support writing non-UTF-8 byte sequences"` even though further writes could complete the codepoint. This is currently a rare occurence since the [linewritershim](2c56ea38b0/library/std/src/io/buffered/linewritershim.rs) implementation flushes complete lines immediately and buffers up to 1024 bytes for incomplete lines. It can still happen as described in #83258.
The problem will become more pronounced once the developer can switch stdout/stderr from line-buffered to block-buffered or immediate when the changes in the "Switchable buffering for Stdout" pull request (#78515) get merged.
# Patch description
If there is at least one valid UTF-8 codepoint all valid UTF-8 is passed through to the extracted `write_valid_utf8_to_console()` fn. The new code only comes into play if `write()` is being passed a short byte slice comprising an incomplete UTF-8 codepoint. In this case up to three bytes are buffered in the `IncompleteUtf8` struct associated with `Stdout` / `Stderr`. The bytes are accepted one at a time. As soon as an error can be detected `io::ErrorKind::InvalidData, "Windows stdio in console mode does not support writing non-UTF-8 byte sequences"` is returned. Once a complete UTF-8 codepoint is received it is passed to the `write_valid_utf8_to_console()` and the buffer length is set to zero.
Calling `flush()` will neither error nor write anything if an incomplete codepoint is present in the buffer.
# Tests
Currently there are no Windows-specific tests for console writing code at all. Writing (regression) tests for this problem is a bit challenging since unit tests and UI tests don't run in a console and suddenly popping up another console window might be surprising to developers running the testsuite and it might not work at all in CI builds. To just test the new functionality in unit tests the code would need to be refactored. Some guidance on how to proceed would be appreciated.
# Public API changes
* `std::str::verifications::utf8_char_width()` would be exposed as `std::str::utf8_char_width()` behind the "str_internals" feature gate.
# Related issues
* Fixes#83258.
* PR #78515 will exacerbate the problem.
# Open questions
* Add tests?
* Squash into one commit with better commit message?
Stabilize `UnsafeCell::raw_get()`
This PR stabilizes the associated function `UnsafeCell::raw_get()`. The FCP has [already completed](https://github.com/rust-lang/rust/issues/66358#issuecomment-899095068). While there was some discussion about the naming after the close of the FCP, it looks like people have agreed on this name. Still, it would probably be best if a `libs-api` member had a look at this and stated whether more discussion is needed.
While I was at it, I added some tests for `UnsafeCell`, because there were barely any.
Closes#66358.
Use the return value of readdir_r() instead of errno
POSIX says:
> If successful, the readdir_r() function shall return zero; otherwise,
> an error number shall be returned to indicate the error.
But we were previously using errno instead of the return value. This
led to issue #86649.
POSIX says:
> If successful, the readdir_r() function shall return zero; otherwise,
> an error number shall be returned to indicate the error.
But we were previously using errno instead of the return value. This
led to issue #86649.
Add `TcpStream::set_linger` and `TcpStream::linger`
Adds methods for getting/setting the `SO_LINGER` option on TCP sockets. Behavior is consistent across Unix and Windows.
r? `@joshtriplett` (I noticed you've been reviewing net related PRs)
Document `std::env::current_exe` possible rename behaviour
It might not be obvious that the "path of the current running executable" may (or may not) imply "at the time it was loaded".
This came up recently in chat so I thought it might be worth documenting.
Add Saturating type (based on Wrapping type)
Tracking #87920
### Unresolved Questions
<!--
Include any open questions that need to be answered before the feature can be
stabilised.
-->
- [x] ~`impl Div for Saturating<T>` falls back on inner integer division - which seems alright?~
- [x] add `saturating_div`? (to respect division by `-1`)
- [x] There is no `::saturating_shl` and `::saturating_shr`. (How to) implement `Shl`, `ShlAssign`, `Shr` and `ShrAssign`?
- [naively](3f7d2ce28f)
- [x] ~`saturating_neg` is only implemented on [signed integer types](https://doc.rust-lang.org/std/?search=saturating_n)~
- [x] Is the implementation copied over from the `Wrapping`-type correct for `Saturating`?
- [x] `Saturating::rotate_left`
- [x] `Saturating::rotate_right`
- [x] `Not`
- [x] `BitXorOr` and `BitXorOrAssign`
- [x] `BitOr` and `BitOrAssign`
- [x] `BitAnd` and `BitAndAssign`
- [x] `Saturating::swap_bytes`
- [x] `Saturating::reverse_bits`
Add `c_size_t` and `c_ssize_t` to `std::os::raw`.
Apparently these aren't guaranteed to be the same, and are merely "always the same in practice" (see https://rust-lang.zulipchat.com/#narrow/stream/136281-t-lang.2Fwg-unsafe-code-guidelines/topic/.60usize.60.20vs.20.60size_t.60).
This is a big footgun, but I suspect it can be alleviated if we expose this and start migrating people to it in advance of any platforms that ever have this as different.
I'll file a tracking issue after this gets some traction.
Stabilise BufWriter::into_parts
The FCP for this has already completed, in #80690.
This was just blocked on #85901 (which changed the name), which is now merged. The original stabilisation MR was #84770 but that has a lot of noise in it, and I also accidentally deleted the branch while trying to tidy up. So here is a new MR. Sorry for the noise.
Closes#80690
Errorkind reorder
I was doing a bit more work in this area and the untidiness of these two orderings bothered me.
The commit messages have the detailed rationale. For your convenience, I c&p them here:
```
io::ErrorKind: rationalise ordering in main enum
It is useful to keep some coherent structure to this ordering. In
particular, Other and Uncategorized should be next to each other, at
the end.
Also it seems to make sense to treat UnexpectedEof and OutOfMemory
specially, since they are not like the other errors (despite
OutOfMemory also being generatable by some OS errors).
So:
* Move Other to the end, just before Uncategorized
* Move Unsupported to between Interrupted and UnexpectedEof
* Add some comments documenting where to add things
```
```
io::Error: alphabeticise the match in as_str()
There was no rationale for the previous ordering.
```
r? kennytm since that's who rust-highfive picked before, in #88294 which I accidentally closed.
It is useful to keep some coherent structure to this ordering. In
particular, Other and Uncategorized should be next to each other, at
the end.
Also it seems to make sense to treat UnexpectedEof and OutOfMemory
specially, since they are not like the other errors (despite
OutOfMemory also being generatable by some OS errors).
So:
* Move Other to the end, just before Uncategorized
* Move Unsupported to between Interrupted and UnexpectedEof
* Add some comments documenting where to add things
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Implement `AsFd` etc. for `UnixListener`.
Implement `AsFd`, `From<OwnedFd>`, and `Into<OwnedFd>` for
`UnixListener`. This is a follow-up to #87329.
r? `@joshtriplett`
add file_prefix method to std::path
This is an initial implementation of `std::path::Path::file_prefix`. It is effectively a "left" variant of the existing [`file_stem`](https://doc.rust-lang.org/std/path/struct.Path.html#method.file_stem) method. An illustration of the difference is
```rust
use std::path::Path;
let path = Path::new("foo.tar.gz");
assert_eq!(path.file_stem(), Some("foo.tar"));
assert_eq!(path.file_prefix(), Some("foo"));
```
In my own development, I generally find I almost always want the prefix, rather than the stem, so I thought it might be best to suggest it's addition to libstd.
Of course, as this is my first contribution, I expect there is probably more work that needs to be done. Additionally, if the libstd team feel this isn't appropriate then so be it.
There has been some [discussion about this on Zulip](https://rust-lang.zulipchat.com/#narrow/stream/219381-t-libs/topic/file_lstem/near/238076313) and a user there suggested I open a PR to see whether someone in the libstd team thinks it is worth pursuing.
where available use AtomicU{64,128} instead of mutex for Instant backsliding protection
This decreases the overhead of backsliding protection on x86 systems with unreliable TSC, e.g. windows. And on aarch64 systems where 128bit atomics are available.
The following benchmarks were taken on x86_64 linux though by overriding `actually_monotonic()`, the numbers may look different on other platforms
```
# actually_monotonic() == true
test time::tests::instant_contention_01_threads ... bench: 44 ns/iter (+/- 0)
test time::tests::instant_contention_02_threads ... bench: 44 ns/iter (+/- 0)
test time::tests::instant_contention_04_threads ... bench: 44 ns/iter (+/- 0)
test time::tests::instant_contention_08_threads ... bench: 44 ns/iter (+/- 0)
test time::tests::instant_contention_16_threads ... bench: 44 ns/iter (+/- 0)
# 1x AtomicU64
test time::tests::instant_contention_01_threads ... bench: 65 ns/iter (+/- 0)
test time::tests::instant_contention_02_threads ... bench: 157 ns/iter (+/- 20)
test time::tests::instant_contention_04_threads ... bench: 281 ns/iter (+/- 53)
test time::tests::instant_contention_08_threads ... bench: 555 ns/iter (+/- 77)
test time::tests::instant_contention_16_threads ... bench: 883 ns/iter (+/- 107)
# mutex
test time::tests::instant_contention_01_threads ... bench: 60 ns/iter (+/- 2)
test time::tests::instant_contention_02_threads ... bench: 770 ns/iter (+/- 231)
test time::tests::instant_contention_04_threads ... bench: 1,347 ns/iter (+/- 45)
test time::tests::instant_contention_08_threads ... bench: 2,693 ns/iter (+/- 114)
test time::tests::instant_contention_16_threads ... bench: 5,244 ns/iter (+/- 487)
```
Since I don't have an arm machine with 128bit atomics I wasn't able to benchmark the AtomicU128 implementation.
I/O safety.
Introduce `OwnedFd` and `BorrowedFd`, and the `AsFd` trait, and
implementations of `AsFd`, `From<OwnedFd>` and `From<T> for OwnedFd`
for relevant types, along with Windows counterparts for handles and
sockets.
Tracking issue: <https://github.com/rust-lang/rust/issues/87074>
RFC: <https://github.com/rust-lang/rfcs/blob/master/text/3128-io-safety.md>
Highlights:
- The doc comments at the top of library/std/src/os/unix/io/mod.rs and library/std/src/os/windows/io/mod.rs
- The new types and traits in library/std/src/os/unix/io/fd.rs and library/std/src/os/windows/io/handle.rs
- The removal of the `RawHandle` struct the Windows impl, which had the same name as the `RawHandle` type alias, and its functionality is now folded into `Handle`.
Managing five levels of wrapping (File wraps sys::fs::File wraps sys::fs::FileDesc wraps OwnedFd wraps RawFd, etc.) made for a fair amount of churn and verbose as/into/from sequences in some places. I've managed to simplify some of them, but I'm open to ideas here.
r? `@joshtriplett`
Add fast path for Path::cmp that skips over long shared prefixes
```
# before
test path::tests::bench_path_cmp_fast_path_buf_sort ... bench: 60,811 ns/iter (+/- 865)
test path::tests::bench_path_cmp_fast_path_long ... bench: 6,459 ns/iter (+/- 275)
test path::tests::bench_path_cmp_fast_path_short ... bench: 1,777 ns/iter (+/- 34)
# after
test path::tests::bench_path_cmp_fast_path_buf_sort ... bench: 38,140 ns/iter (+/- 211)
test path::tests::bench_path_cmp_fast_path_long ... bench: 1,471 ns/iter (+/- 24)
test path::tests::bench_path_cmp_fast_path_short ... bench: 1,106 ns/iter (+/- 9)
```
