rust/crates
Aleksey Kladov a5c333c3ed Fix yet another parser infinite loop
This commit is an example of fixing a common parser error: infinite
loop due to error recovery.

This error typically happens when we parse a list of items and fail to
parse a specific item at the current position.

One choices is to skip a token and try to parse a list item at the
next position. This is a good, but not universal, default. When
parsing a list of arguments in a function call, you, for example,
don't want to skip over `fn`, because it's most likely that it is a
function declaration, and not a mistyped arg:

```
fn foo() {
    quux(1, 2

fn bar() {
}
```

Another choice is to bail out of the loop immediately, but it isn't
perfect either: sometimes skipping over garbage helps:

```
quux(1, foo:, 92) // should skip over `:`, b/c that's part of `foo::bar`
```

In general, parser tries to balance these two cases, though we don't
have a definitive strategy yet.

However, if the parser accidentally neither skips over a token, nor
breaks out of the loop, then it becomes stuck in the loop infinitely
(there's an internal counter to self-check this situation and panic
though), and that's exactly what is demonstrated by the test.

To fix such situation, first of all, add the test case to tests/data/parser/{err,fuzz-failures}.

Then, run

```
RUST_BACKTRACE=short cargo test --package libsyntax2
````

to verify that parser indeed panics, and to get an idea what grammar
production is the culprit (look for `_list` functions!).

In this case, I see

```
  10: libsyntax2::grammar::expressions::atom::match_arm_list
             at crates/libsyntax2/src/grammar/expressions/atom.rs:309
```

and that's look like it might be a culprit. I verify it by adding
`eprintln!("loopy {:?}", p.current());` and indeed I see that this is
printed repeatedly.

Diagnosing this a bit shows that the problem is that
`pattern::pattern` function does not consume anything if the next
token is `let`. That is a good default to make cases like

```
let
let foo = 92;
```

where the user hasn't typed the pattern yet, to parse in a reasonable
they correctly.

For match arms, pretty much the single thing we expect is a pattern,
so, for a fix, I introduce a special variant of pattern that does not
do recovery.
2018-09-08 19:10:40 +03:00
..
cli kill more reexports 2018-08-25 11:48:59 +03:00
gen_lsp_server use correct workdir for the server 2018-09-05 21:38:43 +03:00
libanalysis Don't overflow when limiting symbol search 2018-09-08 15:39:28 +03:00
libeditor nested mod completion 2018-09-08 01:35:20 +03:00
libsyntax2 Fix yet another parser infinite loop 2018-09-08 19:10:40 +03:00
server Some abstraction around workers 2018-09-08 13:15:01 +03:00
test_utils move 2018-08-25 14:30:54 +03:00
tools default method name to type name 2018-08-31 15:10:37 +03:00