However, I left the ICEs rather than switching to nicer error
reporting there; these cases *should* be detected prior to hitting
trans, and thus ICE'ing here is appropriate.
(So why switch to `eval_const_expr_partial`? Because I want to try to
eliminate all uses of `eval_const_expr` entirely.)
1. Detect and report arithmetic overflow during const-expr eval.
2. Instead `eval_const_expr_partial` returning `Err(String)`, it now
has a dedicated enum of different cases. The main benefit of this
is the ability to pass along an interpretable payload, namely the
two inputs that caused an overlfow.
I attempted to minimize fallout to error output in tests, but some was
unavoidable. Those changes are in a follow-on commit.
This allows computation to proceed and find further errors.
(However, this is also annoying at times when the subsequent errors
are just reporting that a ty_err occurred. I have thoughts on ways to
fix this that I will experiment with separately.)
This only replaces the conditional arith-overflow asserts with
unconditional errors from the guts of const-eval; it does *not*
attempt to sanely handle such errors e.g. with a nice error message
from `rustc`. So the same test that led me to add this commit are
still failing, and must be addressed.
(The bug was in `impl RandomAccessIterator for Rev`; it may or may not
have been innocuous, depending on what guarantees one has about the
behavior of `idx` for the underlying iterator.)
Regarding the `rand` changes: It is unfortunate that Wrapping(T) does
not support the `+=` operator. We may want to try to fix that before
1.0 to make porting code like this palatable.
Regarding `std::rand`, just arith-overflow in first example from
`std::rand::random()` doc.
* `core::num`: adjust `UnsignedInt::is_power_of_two`,
`UnsignedInt::next_power_of_two`, `Int::pow`.
In particular for `Int::pow`: (1.) do not panic when `base`
overflows if `acc` never observes the overflowed `base`, and (2.)
if `acc` does observe the overflowed `base`, make sure we only
panic if we would have otherwise (e.g. during a computation of
`base * base`).
* also in `core::num`: avoid underflow during computation of `uint::MAX`.
* `std::num`: adjust tests `uint::test_uint_from_str_overflow`,
`uint::test_uint_to_str_overflow`, `strconv`
* `coretest::num`: adjust `test::test_int_from_str_overflow`.
* The error patterns had a typo.
* Our current constant evaluation would silently allow the overflow
(filed as Issue 22531).
* The overflowing-mul test was accidentally doing addition instead of
multiplication.
* `collections::btree::node`: accommodate (transient) underflow.
* `collections::btree::map`: avoid underflow during `fn next`
for `BTreeMap::range` methods.
* `collections::slice`: note that pnkfelix deliberately used
`new_pos_wrapping` only once; the other cases of arithmetic do not
over- nor underflow, which is a useful property to leave implicitly
checked/documented via the remaining calls to `fn new_pos(..)`.
* `collections::vec_deque` applied wrapping ops (somewhat blindly)
to two implementation methods, and many tests.
* `std::collections:#️⃣:table` : Use `OverflowingOps` trait to
track overflow during `calculate_offsets` and `calculate_allocation`
functions.
These return the result of the operation *plus* an overflow/underflow bit.
This can make it easier to write operations where you want to chain
some arithmetic together, but also want to return a flag signalling if
overflow every occurred.
During my clean-up of rebase errors, I took the opportunity to implement
parse_opt_bool so that it isn't identical to parse_bool wrapped in
`Some`.
parse_opt_bool considers no value to be true, a value of 'y', 'yes' or
'on' to be true and 'n', 'no' or 'off' to be false. All other values are
an error.
Many of the core rust libraries have places that rely on integer
wrapping behaviour. These places have been altered to use the wrapping_*
methods:
* core:#️⃣:sip - A number of macros
* core::str - The `maximal_suffix` method in `TwoWaySearcher`
* rustc::util::nodemap - Implementation of FnvHash
* rustc_back::sha2 - A number of macros and other places
* rand::isaac - Isaac64Rng, changed to use the Wrapping helper type
Some places had "benign" underflow. This is when underflow or overflow
occurs, but the unspecified value is not used due to other conditions.
* collections::bit::Bitv - underflow when `self.nbits` is zero.
* collections:#️⃣:{map,table} - Underflow when searching an empty
table. Did cause undefined behaviour in this case due to an
out-of-bounds ptr::offset based on the underflowed index. However the
resulting pointers would never be read from.
* syntax::ext::deriving::encodable - Underflow when calculating the
index of the last field in a variant with no fields.
These cases were altered to avoid the underflow, often by moving the
underflowing operation to a place where underflow could not happen.
There was one case that relied on the fact that unsigned arithmetic and
two's complement arithmetic are identical with wrapping semantics. This
was changed to use the wrapping_* methods.
Finally, the calculation of variant discriminants could overflow if the
preceeding discriminant was `U64_MAX`. The logic in `rustc::middle::ty`
for this was altered to avoid the overflow completely, while the
remaining places were changed to use wrapping methods. This is because
`rustc::middle::ty::enum_variants` now throws an error when the
calculated discriminant value overflows a `u64`.
This behaviour can be triggered by the following code:
```
enum Foo {
A = U64_MAX,
B
}
```
This commit also implements the remaining integer operators for
Wrapped<T>.
Adds overflow checking to integer addition, multiplication, and subtraction
when `-Z force-overflow-checks` is true, or if `--cfg ndebug` is not passed to
the compiler. On overflow, it panics with `arithmetic operation overflowed`.
Also adds `overflowing_add`, `overflowing_sub`, and `overflowing_mul`
intrinsics for doing unchecked arithmetic.
[breaking-change]
Namely, I have been annoyed in the past when I have done
`RUST_BACKTRACE=1 make check` only to discover (again) that such a
trick causes this test to fail, because it assumes that the
`RUST_BACKTRACE` environment variable is not set.
This is a series of individual but correlated changes to the metadata format. The changes are significant enough that it (finally) bumps the metadata encoding version. In brief, they altogether reduce the total size of stage1 binaries by 27% (!!!!). Almost every low-hanging fruit has been considered and fixed; see the individual commits for details.
Detailed library (not just metadata) size changes for x86_64-unknown-linux-gnu stage1 binaries (baseline being 3a96d6a981):
````
before after delta path
--------- --------- ------ --------------------------------
1706146 1050412 38.4% liballoc-4e7c5e5c.rlib
398576 152454 61.8% libarena-4e7c5e5c.rlib
71441 56892 20.4% libarena-4e7c5e5c.so
14424754 5084102 64.8% libcollections-4e7c5e5c.rlib
39143186 14743118 62.3% libcore-4e7c5e5c.rlib
195574 188150 3.8% libflate-4e7c5e5c.rlib
153123 152603 0.3% libflate-4e7c5e5c.so
477152 215262 54.9% libfmt_macros-4e7c5e5c.rlib
77728 66601 14.3% libfmt_macros-4e7c5e5c.so
1216936 684104 43.8% libgetopts-4e7c5e5c.rlib
207846 181116 12.9% libgetopts-4e7c5e5c.so
349722 147530 57.8% libgraphviz-4e7c5e5c.rlib
60196 49197 18.3% libgraphviz-4e7c5e5c.so
729842 259906 64.4% liblibc-4e7c5e5c.rlib
349358 247014 29.3% liblog-4e7c5e5c.rlib
88878 83163 6.4% liblog-4e7c5e5c.so
1968508 732840 62.8% librand-4e7c5e5c.rlib
1968204 696326 64.6% librbml-4e7c5e5c.rlib
283207 206589 27.1% librbml-4e7c5e5c.so
72369394 46401230 35.9% librustc-4e7c5e5c.rlib
11941372 10498483 12.1% librustc-4e7c5e5c.so
2717894 1983272 27.0% librustc_back-4e7c5e5c.rlib
501900 464176 7.5% librustc_back-4e7c5e5c.so
15058 12588 16.4% librustc_bitflags-4e7c5e5c.rlib
4008268 2961912 26.1% librustc_borrowck-4e7c5e5c.rlib
837550 785633 6.2% librustc_borrowck-4e7c5e5c.so
6473348 6095470 5.8% librustc_driver-4e7c5e5c.rlib
1448785 1433945 1.0% librustc_driver-4e7c5e5c.so
95483688 94779704 0.7% librustc_llvm-4e7c5e5c.rlib
43516815 43487809 0.1% librustc_llvm-4e7c5e5c.so
938140 817236 12.9% librustc_privacy-4e7c5e5c.rlib
182653 176563 3.3% librustc_privacy-4e7c5e5c.so
4390288 3543284 19.3% librustc_resolve-4e7c5e5c.rlib
872981 831824 4.7% librustc_resolve-4e7c5e5c.so
18176426 14795426 18.6% librustc_trans-4e7c5e5c.rlib
3657354 3480026 4.8% librustc_trans-4e7c5e5c.so
16815076 13868862 17.5% librustc_typeck-4e7c5e5c.rlib
3274439 3123898 4.6% librustc_typeck-4e7c5e5c.so
21372308 14890582 30.3% librustdoc-4e7c5e5c.rlib
4501971 4172202 7.3% librustdoc-4e7c5e5c.so
8055028 2951044 63.4% libserialize-4e7c5e5c.rlib
958101 710016 25.9% libserialize-4e7c5e5c.so
30810208 15160648 50.8% libstd-4e7c5e5c.rlib
6819003 5967485 12.5% libstd-4e7c5e5c.so
58850950 31949594 45.7% libsyntax-4e7c5e5c.rlib
9060154 7882423 13.0% libsyntax-4e7c5e5c.so
1474310 1062102 28.0% libterm-4e7c5e5c.rlib
345577 323952 6.3% libterm-4e7c5e5c.so
2827854 1643056 41.9% libtest-4e7c5e5c.rlib
517811 452519 12.6% libtest-4e7c5e5c.so
2274106 1761240 22.6% libunicode-4e7c5e5c.rlib
--------- --------- ------ --------------------------------
499359187 363465583 27.2% total
````
Some notes:
* Uncompressed metadata compacts very well. It is less visible for compressed metadata but still it achieves about 5~10% reduction.
* *Every* commit is designed to reduce the metadata in one way. There is absolutely no negative impact associated to changes (that's why the table above doesn't contain a minus delta).
* I've confirmed that this compiles through `make all`, making it almost correct. Other platforms have to be tested though.
* Oh, I'll rebase this as soon as I have spare time, but I guess this needs an extensive review anyway.
* I haven't rigorously checked the encoder and decoder performance. I tried to minimize the impact (some encodings are actually simpler than the original), but I'm not sure.
Fixes#2743, #9303 (partially) and #21482.
Fix the return type in the comments.
An old commit 082bfde412 ("Fallout of std::str stabilization") removed
the example of FromStr::from_str(), this commit adds it back. But
the example of StrExt::parse() is still kept with an additinal note.
Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
Previously every auto-serialized tags are strongly typed. However
this is not strictly required, and instead it can be exploited
to provide the optimal encoding for smaller integers. This commit
repurposes `EsI8`/`EsU8` through `EsI64`/`EsU64` tags to represent
*any* integers with given ranges: It is now possible to encode
`42u64` as two bytes `EsU8 0x2a`, for example.
There are some limitations:
* It does not apply to non-auto-serialized tags for obvious reasons.
Fortunately, we have already eliminated the biggest source of
such tag in favor of auto-serialized tags: `tag_table_id`.
* Bigger tags cannot be used to represent smaller types.
* Signed tags and unsigned tags do not mix.
This avoids a biggish eight-byte `tag_table_id` tag in favor of
autoserialized integer tags, which are smaller and can be later
used to encode them in the optimal number of bytes. `NodeId` was
u32 after all.
Previously:
<------------- len1 -------------->
tag_table_* <len1> tag_table_id 88 <nodeid in 8 bytes>
tag_table_val <len2> <actual data>
<-- len2 --->
Now:
<--------------- len --------------->
tag_table_* <len> U32 <nodeid in 4 bytes> <actual data>
We try to move the data when the length can be encoded in
the much smaller number of bytes. This interferes with indices and
type abbreviations however, so this commit introduces a public
interface to get and mark a "stable" (i.e. not affected by
relaxation) position of the current pointer.
The relaxation logic only moves a small data, currently at most
256 bytes, as moving the data can be costly. There might be
further opportunities to allow more relaxation by moving fields
around, which I didn't seriously try.
They replace the existing `EsEnumVid`, `EsVecLen` and `EsMapLen`
tags altogether; the meaning of them can be easily inferred
from the enclosing tag. It also has an added benefit of
encodings for smaller variant ids or lengths being more compact
(5 bytes to 2 bytes).
For the reference, while it is designed to be selectively enabled,
it was essentially enabled throughout every snapshot and nightly
as far as I can tell. This makes the usefulness of `EsLabel` itself
questionable, as it was quite rare that `EsLabel` broke the build.
It had consumed about 20~30% of metadata (!) and so this should be
a huge win.
It doesn't serve any useful purpose. It *might* be useful when
there are some tags that are generated by `Encodable` and
not delimited by any tags, but IIUC it's not the case.
Previous:
<-------------------- len1 ------------------->
EsEnum <len1> EsEnumVid <vid> EsEnumBody <len2> <arg1> <arg2>
<--- len2 -->
Now:
<----------- len1 ---------->
EsEnum <len1> EsEnumVid <vid> <arg1> <arg2>
Many auto-serialization tags are fixed-size (note: many ordinary
tags are also fixed-size but for now this commit ignores them),
so having an explicit length is a waste. This moves any
auto-serialization tags with an implicit length before other tags,
so a test for them is easy. A preliminary experiment shows this
has at least 1% gain over the status quo.
EBML tags are encoded in a variable-length unsigned int (vuint),
which is clever but causes some tags to be encoded in two bytes
while there are really about 180 tags or so. Assuming that there
wouldn't be, say, over 1,000 tags in the future, we can use much
more efficient encoding scheme. The new scheme should support
at most 4,096 tags anyway.
This also flattens a scattered tag namespace (did you know that
0xa9 is followed by 0xb0?) and makes a room for autoserialized tags
in 0x00 through 0x1f.
They are, with a conjunction of `start_tag` and `end_tag`, commonly
used to write a document with a binary data of known size. However
the use of `start_tag` makes the length always 4 bytes long, which
is almost not optimal (requiring the relaxation step to remedy).
Directly using `wr_tagged_*` methods is better for both readability
and resulting metadata size.
* count_ones/zeros, trailing_ones/zeros return u32, not usize
* rotate_left/right take u32, not usize
* RADIX, MANTISSA_DIGITS, DIGITS, BITS, BYTES are u32, not usize
Doesn't touch pow because there's another PR for it.
cc https://github.com/rust-lang/rust/issues/22240
r? @Gankro
The API this exposes is a little strange (being attached to `static`s),
so it makes sense to conservatively feature gate it. If it is highly
popular, it is possible to reverse this gating.
* count_ones/zeros, trailing_ones/zeros return u32, not usize
* rotate_left/right take u32, not usize
* RADIX, MANTISSA_DIGITS, DIGITS, BITS, BYTES are u32, not usize
Doesn't touch pow because there's another PR for it.
[breaking-change]
* The lint visitor's visit_ty method did not recurse, and had a
reference to the now closed#10894
* The newly enabled recursion has only affected the `deprectated` lint
which now detects uses of deprecated items in trait impls and
function return types
* Renamed some references to `CowString` and `CowVec` to `Cow<str>` and
`Cow<[T]>`, respectively, which appear outside of the crate which
defines them
* Replaced a few instances of `InvariantType<T>` with
`PhantomData<Cell<T>>`
* Disabled the `deprecated` lint in several places that
reference/implement traits on deprecated items which will get cleaned
up in the future
* Disabled the `exceeding_bitshifts` lint for
compile-fail/huge-array-simple test so it doesn't shadow the expected
error on 32bit systems
* Unfortunately, this means that if a library declares
`#![deny(deprecated)]` and marks anything as deprecated, it will have
to disable the lint for any uses of said item, e.g. any impl the now
deprecated item
For any library that denies deprecated items but has deprecated items
of its own, this is a [breaking-change]
This commits blanket marks the API of the `std::process` module as `#[stable]`.
The module's API is very similar to the old `std::old_io::process` API and has
generally had quite a bit of time to bake both before and after the new module
landed.
Remove the synthetic \"region bound\" from closures and instead update how
type-outlives works for closure types so that it ensures that all upvars
outlive the region in question. This gives the same guarantees but
without introducing artificial regions (and gives better error messages
to boot). This is refactoring towards #3696.
r? @pnkfelix
