squiid/rust
1
0
Fork 0
Code Issues Pull requests Projects Activity

Edit librustc_lexer top-level docs

Browse source 
Add link to librustc_parse::lexer
This commit is contained in:
pierwill 2020-06-28 16:32:28 -07:00
parent 2f517ce6f2
commit 36e50a0fb3
1 changed files with 5 additions and 4 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

9
src/librustc_lexer/src/lib.rs
View file

@ -6,17 +6,18 @@
//! produces simple tokens which are a pair of type-tag and a bit of original text,
//! and does not report errors, instead storing them as flags on the token.
//!
//! Tokens produced by this lexer are not yet ready for parsing the Rust syntax,
//! for that see `librustc_parse::lexer`, which converts this basic token stream
//! Tokens produced by this lexer are not yet ready for parsing the Rust syntax.
//! For that see [`librustc_parse::lexer`], which converts this basic token stream
//! into wide tokens used by actual parser.
//!
//! The purpose of this crate is to convert raw sources into a labeled sequence
//! of well-known token types, so building an actual Rust token stream will
//! be easier.
//!
//! Main entity of this crate is [`TokenKind`] enum which represents common
//! The main entity of this crate is the [`TokenKind`] enum which represents common
//! lexeme types.
//!
//! [`librustc_parse::lexer`]: ../rustc_parse/lexer/index.html
// We want to be able to build this crate with a stable compiler, so no
// `#![feature]` attributes should be added.