Commit graph

681 commits

Author SHA1 Message Date
Aleksey Kladov
722706fe41 get rid of commandspeck 2018-09-16 02:12:53 +03:00
bors[bot]
3993bb4de9 Merge #67
67: Salsa r=matklad a=matklad

The aim of this PR is to transition from rather ad-hock FileData and ModuleMap caching strategy to something resembling a general-purpose red-green engine. 

Ideally, we shouldn't recompute ModuleMap at all, unless the set of mod decls or files changes.



Co-authored-by: Aleksey Kladov <aleksey.kladov@gmail.com>
2018-09-15 21:11:25 +00:00
Aleksey Kladov
fcdf3a52b4 everysalsa 2018-09-16 00:00:05 +03:00
Aleksey Kladov
e69ff21207 kill old module_map 2018-09-16 00:00:05 +03:00
Aleksey Kladov
3ebeb0db8d move readonly source to module tree descr 2018-09-16 00:00:05 +03:00
Aleksey Kladov
58674dc3c4 ModuleTreeDescriptor 2018-09-16 00:00:05 +03:00
Aleksey Kladov
d59413c895 yet another db api 2018-09-16 00:00:05 +03:00
Aleksey Kladov
0d7b1e442d minor 2018-09-16 00:00:05 +03:00
Aleksey Kladov
47be3a3a24 renames 2018-09-16 00:00:05 +03:00
Aleksey Kladov
8c737255ff use salsa for new module map 2018-09-16 00:00:05 +03:00
Aleksey Kladov
60fdfec327 eager invalidation 2018-09-16 00:00:05 +03:00
Aleksey Kladov
cecc7ad5b2 be generic over data 2018-09-16 00:00:05 +03:00
Aleksey Kladov
8cf9c27196 generic salsa algo 2018-09-16 00:00:05 +03:00
Aleksey Kladov
0e493160c0 store params in the graph 2018-09-16 00:00:05 +03:00
Aleksey Kladov
907d44a751 any-cache 2018-09-16 00:00:05 +03:00
Aleksey Kladov
dbdf72e2e2 fix dep tracking 2018-09-16 00:00:05 +03:00
Aleksey Kladov
c81d0d51bf add deps tracking 2018-09-16 00:00:05 +03:00
Aleksey Kladov
db14b4270c Add simplisitc global modification caching 2018-09-16 00:00:05 +03:00
Aleksey Kladov
3ae3b3eb06 initial query tracing 2018-09-16 00:00:05 +03:00
Aleksey Kladov
99d02fe583 start query-based modules 2018-09-16 00:00:05 +03:00
bors[bot]
2a56b5c4f0 Merge #69
69: Incremental reparsing for single tokens  r=matklad a=darksv

Implement incremental reparsing for `WHITESPACE`, `COMMENT`, `DOC_COMMENT`, `IDENT`, `STRING` and `RAW_STRING`. This allows to avoid reparsing whole blocks when a change was made only within these tokens.

Co-authored-by: darksv <darek969-12@o2.pl>
2018-09-15 20:57:06 +00:00
darksv
ab00639032 independent tests for incremental reparsing of blocks and leaves 2018-09-15 17:05:08 +02:00
darksv
46cee0415c move reparsing tests 2018-09-15 14:35:30 +02:00
darksv
16ad5384f0 commit missing file 2018-09-15 13:42:10 +02:00
darksv
a29211918b create separated mod for reparsing functionality 2018-09-15 13:35:55 +02:00
bors[bot]
6ee4c287f9 Merge #71
71: Support for unions r=matklad a=darksv

Fixes #70 

Co-authored-by: darksv <darek969-12@o2.pl>
2018-09-14 22:46:18 +00:00
darksv
d825cffe3b adjust trailing newline 2018-09-14 23:45:19 +02:00
darksv
ecbfe68bf4 add missing files with inline tests 2018-09-14 23:33:29 +02:00
darksv
100968b689 Support for unions 2018-09-14 22:51:12 +02:00
darksv
bc94bf95ce correctly handle IDENTs when changed to contextual keywords 2018-09-14 19:26:48 +02:00
darksv
c300135322 create leaf directly without calling the parser 2018-09-14 19:23:10 +02:00
darksv
4356240fa4 Incremental reparsing for single tokens (WHITESPACE, COMMENT, DOC_COMMENT, IDENT, STRING, RAW_STRING) 2018-09-13 23:25:05 +02:00
Aleksey Kladov
b6f8037a6f don't get stuck in slice patterns 2018-09-12 11:26:52 +03:00
Aleksey Kladov
ccc75675b6 correctly setup path-map for fs-changes 2018-09-12 11:19:19 +03:00
bors[bot]
e240360ee2 Merge #68
68: Implement incremental reparsing for remaining braced blocks r=matklad a=darksv

Fixes #66

Co-authored-by: darksv <darek969-12@o2.pl>
2018-09-11 07:32:36 +00:00
darksv
d0cfeb4f16 Do not reparse token tree when it is not delimited by braces 2018-09-10 23:21:16 +02:00
darksv
64d07c1bd4 Implement reparsing for remaining blocks 2018-09-10 20:14:09 +02:00
Aleksey Kladov
505895a25f store file rsovler 2018-09-10 12:57:40 +03:00
bors[bot]
4f64709666 Merge #65
65: simplify r=matklad a=matklad



Co-authored-by: Aleksey Kladov <aleksey.kladov@gmail.com>
2018-09-08 16:24:24 +00:00
Aleksey Kladov
bae5938682 note about symbol search 2018-09-08 19:23:52 +03:00
Aleksey Kladov
f19a82beac simplify 2018-09-08 19:16:11 +03:00
bors[bot]
8c3720d4b8 Merge #64
64: Add fuzz test-case with a fix r=matklad a=matklad



Co-authored-by: Aleksey Kladov <aleksey.kladov@gmail.com>
2018-09-08 16:11:31 +00:00
Aleksey Kladov
a5c333c3ed Fix yet another parser infinite loop
This commit is an example of fixing a common parser error: infinite
loop due to error recovery.

This error typically happens when we parse a list of items and fail to
parse a specific item at the current position.

One choices is to skip a token and try to parse a list item at the
next position. This is a good, but not universal, default. When
parsing a list of arguments in a function call, you, for example,
don't want to skip over `fn`, because it's most likely that it is a
function declaration, and not a mistyped arg:

```
fn foo() {
    quux(1, 2

fn bar() {
}
```

Another choice is to bail out of the loop immediately, but it isn't
perfect either: sometimes skipping over garbage helps:

```
quux(1, foo:, 92) // should skip over `:`, b/c that's part of `foo::bar`
```

In general, parser tries to balance these two cases, though we don't
have a definitive strategy yet.

However, if the parser accidentally neither skips over a token, nor
breaks out of the loop, then it becomes stuck in the loop infinitely
(there's an internal counter to self-check this situation and panic
though), and that's exactly what is demonstrated by the test.

To fix such situation, first of all, add the test case to tests/data/parser/{err,fuzz-failures}.

Then, run

```
RUST_BACKTRACE=short cargo test --package libsyntax2
````

to verify that parser indeed panics, and to get an idea what grammar
production is the culprit (look for `_list` functions!).

In this case, I see

```
  10: libsyntax2::grammar::expressions::atom::match_arm_list
             at crates/libsyntax2/src/grammar/expressions/atom.rs:309
```

and that's look like it might be a culprit. I verify it by adding
`eprintln!("loopy {:?}", p.current());` and indeed I see that this is
printed repeatedly.

Diagnosing this a bit shows that the problem is that
`pattern::pattern` function does not consume anything if the next
token is `let`. That is a good default to make cases like

```
let
let foo = 92;
```

where the user hasn't typed the pattern yet, to parse in a reasonable
they correctly.

For match arms, pretty much the single thing we expect is a pattern,
so, for a fix, I introduce a special variant of pattern that does not
do recovery.
2018-09-08 19:10:40 +03:00
Aleksey Kladov
3ab9f4ad7f Add fuzz failures dir 2018-09-08 18:42:59 +03:00
Aleksey Kladov
ba4a697d8c move fuzz-invariants to the library 2018-09-08 18:34:41 +03:00
bors[bot]
a60b9ad963 Merge #63
63: Add trivial fuzzer for parser r=matklad a=killercup

As described in #61, fuzz testing some parts of this would be ~~fun~~
helpful. So, I started with the most trivial fuzzer I could think of:
Put random stuff into File::parse and see what happens.

To speed things up, I also did

    cp src/**/*.rs fuzz/corpus/parser/

in the `crates/libsyntax2/` directory (running the fuzzer once will
generate the necessary directories).

Co-authored-by: Pascal Hertleif <killercup@gmail.com>
2018-09-08 15:20:46 +00:00
Pascal Hertleif
a37cd5ad43 Add trivial fuzzer for parser
As described in #61, fuzz testing some parts of this would be ~~fun~~
helpful. So, I started with the most trivial fuzzer I could think of:
Put random stuff into File::parse and see what happens.

To speed things up, I also did

    cp src/**/*.rs fuzz/corpus/parser/

in the `crates/libsyntax2/` directory (running the fuzzer once will
generate the necessary directories).
2018-09-08 16:55:53 +02:00
Aleksey Kladov
df05c5c3e2 Don't overflow when limiting symbol search 2018-09-08 15:39:28 +03:00
Aleksey Kladov
7daaddb2ac Some abstraction around workers 2018-09-08 13:15:01 +03:00
Aleksey Kladov
326ffcefe0 Deal with deadlocks in a more principaled way 2018-09-08 12:36:02 +03:00