rust/crates/ra_parser
bors[bot] ce694ae118 Merge #1328
1328: Change TokenSource to iteration based r=matklad a=edwin0cheng

This PR change the `TokenSource` trait from random access to be an iteration based trait:
```rust

/// `TokenSource` abstracts the source of the tokens parser operates one.
///
/// Hopefully this will allow us to treat text and token trees in the same way!
pub trait TokenSource {
    fn current(&self) -> Token;

    /// Lookahead n token
    fn lookahead_nth(&self, n: usize) -> Token;

    /// bump cursor to next token
    fn bump(&mut self);

    /// Is the current token a specified keyword?
    fn is_keyword(&self, kw: &str) -> bool;
}

/// `TokenCursor` abstracts the cursor of `TokenSource` operates one.
#[derive(Debug, Copy, Clone, Eq, PartialEq)]
pub struct Token {
    /// What is the current token?
    pub kind: SyntaxKind,

    /// Is the current token joined to the next one (`> >` vs `>>`).
    pub is_jointed_to_next: bool,
}
```

Note that the refactoring based on this new trait will be separated to incoming PRs

Co-authored-by: Edwin Cheng <edwin0cheng@gmail.com>
2019-05-27 07:28:13 +00:00
..
src Merge #1328 2019-05-27 07:28:13 +00:00
Cargo.toml move parser to a separate crate 2019-02-21 13:27:45 +03:00