auto merge of #16995 : kmcallister/rust/plugin-tutorial, r=alexcrichton

@steveklabnik, are you interested in looking this over?
This commit is contained in:
bors 2014-10-03 07:33:26 +00:00
commit 9a2286d3a1
15 changed files with 473 additions and 19 deletions

1
configure vendored
View file

@ -921,6 +921,7 @@ do
make_dir $h/test/doc-guide-pointers
make_dir $h/test/doc-guide-container
make_dir $h/test/doc-guide-tasks
make_dir $h/test/doc-guide-plugin
make_dir $h/test/doc-rust
done

View file

@ -27,7 +27,7 @@
######################################################################
DOCS := index intro tutorial guide guide-ffi guide-macros guide-lifetimes \
guide-tasks guide-container guide-pointers guide-testing \
guide-runtime complement-bugreport \
guide-runtime guide-plugin complement-bugreport \
complement-lang-faq complement-design-faq complement-project-faq rust \
rustdoc guide-unsafe guide-strings reference

View file

@ -33,7 +33,7 @@ something like:
pandoc --from=markdown --to=html5 --number-sections -o reference.html reference.md
~~~~
(rust.md being the Rust Reference Manual.)
(reference.md being the Rust Reference Manual.)
The syntax for pandoc flavored markdown can be found at:
http://johnmacfarlane.net/pandoc/README.html#pandocs-markdown

View file

@ -39,7 +39,7 @@ representation as a primitive. This allows using Rust `enum`s in FFI where C
`enum`s are also used, for most use cases. The attribute can also be applied
to `struct`s to get the same layout as a C struct would.
[repr]: http://doc.rust-lang.org/rust.html#miscellaneous-attributes
[repr]: reference.html#miscellaneous-attributes
## There is no GC
@ -56,7 +56,7 @@ Types which are [`Sync`][sync] are thread-safe when multiple shared
references to them are used concurrently. Types which are not `Sync` are not
thread-safe, and thus when used in a global require unsafe code to use.
[sync]: http://doc.rust-lang.org/core/kinds/trait.Sync.html
[sync]: core/kinds/trait.Sync.html
### If mutable static items that implement `Sync` are safe, why is taking &mut SHARABLE unsafe?
@ -139,8 +139,8 @@ and explicitly calling the `clone` method. Making user-defined copy operators
explicit surfaces the underlying complexity, forcing the developer to opt-in
to potentially expensive operations.
[copy]: http://doc.rust-lang.org/core/kinds/trait.Copy.html
[clone]: http://doc.rust-lang.org/core/clone/trait.Clone.html
[copy]: core/kinds/trait.Copy.html
[clone]: core/clone/trait.Clone.html
## No move constructors

View file

@ -1,5 +1,14 @@
% The Rust Macros Guide
<div class="unstable-feature">
<b>Warning:</b> There are currently various problems with invoking macros, how
they interact with their environment, and how they are used outside of the
location in which they are defined. Macro definitions are likely to change
slightly in the future. For this reason, they are hidden behind the
<code>macro_rules</code> <a href="reference.html#compiler-features">feature
attribute</a>.
</div>
# Introduction
Functions are the primary tool that programmers can use to build abstractions.
@ -448,6 +457,66 @@ fn main() {
The two `'x` names did not clash, which would have caused the loop
to print "I am never printed" and to run forever.
# Scoping and macro import/export
Macros occupy a single global namespace. The interaction with Rust's system of
modules and crates is somewhat complex.
Definition and expansion of macros both happen in a single depth-first,
lexical-order traversal of a crate's source. So a macro defined at module scope
is visible to any subsequent code in the same module, which includes the body
of any subsequent child `mod` items.
If a module has the `macro_escape` attribute, its macros are also visible in
its parent module after the child's `mod` item. If the parent also has
`macro_escape` then the macros will be visible in the grandparent after the
parent's `mod` item, and so forth.
Independent of `macro_escape`, the `macro_export` attribute controls visibility
between crates. Any `macro_rules!` definition with the `macro_export`
attribute will be visible to other crates that have loaded this crate with
`phase(plugin)`. There is currently no way for the importing crate to control
which macros are imported.
An example:
```rust
# #![feature(macro_rules)]
macro_rules! m1 (() => (()))
// visible here: m1
mod foo {
// visible here: m1
#[macro_export]
macro_rules! m2 (() => (()))
// visible here: m1, m2
}
// visible here: m1
macro_rules! m3 (() => (()))
// visible here: m1, m3
#[macro_escape]
mod bar {
// visible here: m1, m3
macro_rules! m4 (() => (()))
// visible here: m1, m3, m4
}
// visible here: m1, m3, m4
# fn main() { }
```
When this library is loaded with `#[phase(plugin)] extern crate`, only `m2`
will be imported.
# A final note
Macros, as currently implemented, are not for the faint of heart. Even
@ -457,3 +526,10 @@ tricky. Invoking the `log_syntax!` macro can help elucidate intermediate
states, invoking `trace_macros!(true)` will automatically print those
intermediate states out, and passing the flag `--pretty expanded` as a
command-line argument to the compiler will show the result of expansion.
If Rust's macro system can't do what you need, you may want to write a
[compiler plugin](guide-plugin.html) instead. Compared to `macro_rules!`
macros, this is significantly more work, the interfaces are much less stable,
and the warnings about debugging apply ten-fold. In exchange you get the
flexibility of running arbitrary Rust code within the compiler. Syntax
extension plugins are sometimes called "procedural macros" for this reason.

259
src/doc/guide-plugin.md Normal file
View file

@ -0,0 +1,259 @@
% The Rust Compiler Plugins Guide
<div class="unstable-feature">
<p>
<b>Warning:</b> Plugins are an advanced, unstable feature! For many details,
the only available documentation is the <a
href="syntax/index.html"><code>libsyntax</code></a> and <a
href="rustc/index.html"><code>librustc</code></a> API docs, or even the source
code itself. These internal compiler APIs are also subject to change at any
time.
</p>
<p>
For defining new syntax it is often much easier to use Rust's <a
href="guide-macros.html">built-in macro system</a>.
</p>
<p style="margin-bottom: 0">
The code in this document uses language features not covered in the Rust
Guide. See the <a href="reference.html">Reference Manual</a> for more
information.
</p>
</div>
# Introduction
`rustc` can load compiler plugins, which are user-provided libraries that
extend the compiler's behavior with new syntax extensions, lint checks, etc.
A plugin is a dynamic library crate with a designated "registrar" function that
registers extensions with `rustc`. Other crates can use these extensions by
loading the plugin crate with `#[phase(plugin)] extern crate`. See the
[`rustc::plugin`](rustc/plugin/index.html) documentation for more about the
mechanics of defining and loading a plugin.
# Syntax extensions
Plugins can extend Rust's syntax in various ways. One kind of syntax extension
is the procedural macro. These are invoked the same way as [ordinary
macros](guide-macros.html), but the expansion is performed by arbitrary Rust
code that manipulates [syntax trees](syntax/ast/index.html) at
compile time.
Let's write a plugin
[`roman_numerals.rs`](https://github.com/rust-lang/rust/tree/master/src/test/auxiliary/roman_numerals.rs)
that implements Roman numeral integer literals.
```ignore
#![crate_type="dylib"]
#![feature(plugin_registrar)]
extern crate syntax;
extern crate rustc;
use syntax::codemap::Span;
use syntax::parse::token::{IDENT, get_ident};
use syntax::ast::{TokenTree, TTTok};
use syntax::ext::base::{ExtCtxt, MacResult, DummyResult, MacExpr};
use syntax::ext::build::AstBuilder; // trait for expr_uint
use rustc::plugin::Registry;
fn expand_rn(cx: &mut ExtCtxt, sp: Span, args: &[TokenTree])
-> Box<MacResult + 'static> {
static NUMERALS: &'static [(&'static str, uint)] = &[
("M", 1000), ("CM", 900), ("D", 500), ("CD", 400),
("C", 100), ("XC", 90), ("L", 50), ("XL", 40),
("X", 10), ("IX", 9), ("V", 5), ("IV", 4),
("I", 1)];
let text = match args {
[TTTok(_, IDENT(s, _))] => get_ident(s).to_string(),
_ => {
cx.span_err(sp, "argument should be a single identifier");
return DummyResult::any(sp);
}
};
let mut text = text.as_slice();
let mut total = 0u;
while !text.is_empty() {
match NUMERALS.iter().find(|&&(rn, _)| text.starts_with(rn)) {
Some(&(rn, val)) => {
total += val;
text = text.slice_from(rn.len());
}
None => {
cx.span_err(sp, "invalid Roman numeral");
return DummyResult::any(sp);
}
}
}
MacExpr::new(cx.expr_uint(sp, total))
}
#[plugin_registrar]
pub fn plugin_registrar(reg: &mut Registry) {
reg.register_macro("rn", expand_rn);
}
```
Then we can use `rn!()` like any other macro:
```ignore
#![feature(phase)]
#[phase(plugin)]
extern crate roman_numerals;
fn main() {
assert_eq!(rn!(MMXV), 2015);
}
```
The advantages over a simple `fn(&str) -> uint` are:
* The (arbitrarily complex) conversion is done at compile time.
* Input validation is also performed at compile time.
* It can be extended to allow use in patterns, which effectively gives
a way to define new literal syntax for any data type.
In addition to procedural macros, you can define new
[`deriving`](reference.html#deriving)-like attributes and other kinds of
extensions. See
[`Registry::register_syntax_extension`](rustc/plugin/registry/struct.Registry.html#method.register_syntax_extension)
and the [`SyntaxExtension`
enum](http://doc.rust-lang.org/syntax/ext/base/enum.SyntaxExtension.html). For
a more involved macro example, see
[`src/libregex_macros/lib.rs`](https://github.com/rust-lang/rust/blob/master/src/libregex_macros/lib.rs)
in the Rust distribution.
## Tips and tricks
To see the results of expanding syntax extensions, run
`rustc --pretty expanded`. The output represents a whole crate, so you
can also feed it back in to `rustc`, which will sometimes produce better
error messages than the original compilation. Note that the
`--pretty expanded` output may have a different meaning if multiple
variables of the same name (but different syntax contexts) are in play
in the same scope. In this case `--pretty expanded,hygiene` will tell
you about the syntax contexts.
You can use [`syntax::parse`](syntax/parse/index.html) to turn token trees into
higher-level syntax elements like expressions:
```ignore
fn expand_foo(cx: &mut ExtCtxt, sp: Span, args: &[TokenTree])
-> Box<MacResult+'static> {
let mut parser =
parse::new_parser_from_tts(cx.parse_sess(), cx.cfg(), args.to_slice())
let expr: P<Expr> = parser.parse_expr();
```
Looking through [`libsyntax` parser
code](https://github.com/rust-lang/rust/blob/master/src/libsyntax/parse/parser.rs)
will give you a feel for how the parsing infrastructure works.
Keep the [`Span`s](syntax/codemap/struct.Span.html) of
everything you parse, for better error reporting. You can wrap
[`Spanned`](syntax/codemap/struct.Spanned.html) around
your custom data structures.
Calling
[`ExtCtxt::span_fatal`](syntax/ext/base/struct.ExtCtxt.html#method.span_fatal)
will immediately abort compilation. It's better to instead call
[`ExtCtxt::span_err`](syntax/ext/base/struct.ExtCtxt.html#method.span_err)
and return
[`DummyResult`](syntax/ext/base/struct.DummyResult.html),
so that the compiler can continue and find further errors.
The example above produced an integer literal using
[`AstBuilder::expr_uint`](syntax/ext/build/trait.AstBuilder.html#tymethod.expr_uint).
As an alternative to the `AstBuilder` trait, `libsyntax` provides a set of
[quasiquote macros](syntax/ext/quote/index.html). They are undocumented and
very rough around the edges. However, the implementation may be a good
starting point for an improved quasiquote as an ordinary plugin library.
# Lint plugins
Plugins can extend [Rust's lint
infrastructure](reference.html#lint-check-attributes) with additional checks for
code style, safety, etc. You can see
[`src/test/auxiliary/lint_plugin_test.rs`](https://github.com/rust-lang/rust/blob/master/src/test/auxiliary/lint_plugin_test.rs)
for a full example, the core of which is reproduced here:
```ignore
declare_lint!(TEST_LINT, Warn,
"Warn about items named 'lintme'")
struct Pass;
impl LintPass for Pass {
fn get_lints(&self) -> LintArray {
lint_array!(TEST_LINT)
}
fn check_item(&mut self, cx: &Context, it: &ast::Item) {
let name = token::get_ident(it.ident);
if name.get() == "lintme" {
cx.span_lint(TEST_LINT, it.span, "item is named 'lintme'");
}
}
}
#[plugin_registrar]
pub fn plugin_registrar(reg: &mut Registry) {
reg.register_lint_pass(box Pass as LintPassObject);
}
```
Then code like
```ignore
#[phase(plugin)]
extern crate lint_plugin_test;
fn lintme() { }
```
will produce a compiler warning:
```txt
foo.rs:4:1: 4:16 warning: item is named 'lintme', #[warn(test_lint)] on by default
foo.rs:4 fn lintme() { }
^~~~~~~~~~~~~~~
```
The components of a lint plugin are:
* one or more `declare_lint!` invocations, which define static
[`Lint`](rustc/lint/struct.Lint.html) structs;
* a struct holding any state needed by the lint pass (here, none);
* a [`LintPass`](rustc/lint/trait.LintPass.html)
implementation defining how to check each syntax element. A single
`LintPass` may call `span_lint` for several different `Lint`s, but should
register them all through the `get_lints` method.
Lint passes are syntax traversals, but they run at a late stage of compilation
where type information is available. `rustc`'s [built-in
lints](https://github.com/rust-lang/rust/blob/master/src/librustc/lint/builtin.rs)
mostly use the same infrastructure as lint plugins, and provide examples of how
to access type information.
Lints defined by plugins are controlled by the usual [attributes and compiler
flags](reference.html#lint-check-attributes), e.g. `#[allow(test_lint)]` or
`-A test-lint`. These identifiers are derived from the first argument to
`declare_lint!`, with appropriate case and punctuation conversion.
You can run `rustc -W help foo.rs` to see a list of lints known to `rustc`,
including those provided by plugins loaded by `foo.rs`.

View file

@ -1843,9 +1843,9 @@ that page, but the best part is the search bar. Right up at the top, there's
a box that you can enter in a search term. The search is pretty primitive
right now, but is getting better all the time. If you type 'random' in that
box, the page will update to [this
one](http://doc.rust-lang.org/std/index.html?search=random). The very first
one](std/index.html?search=random). The very first
result is a link to
[std::rand::random](http://doc.rust-lang.org/std/rand/fn.random.html). If we
[std::rand::random](std/rand/fn.random.html). If we
click on that result, we'll be taken to its documentation page.
This page shows us a few things: the type signature of the function, some
@ -3723,7 +3723,7 @@ If you use `Rc<T>` or `Arc<T>`, you have to be careful about introducing
cycles. If you have two `Rc<T>`s that point to each other, the reference counts
will never drop to zero, and you'll have a memory leak. To learn more, check
out [the section on `Rc<T>` and `Arc<T>` in the pointers
guide](http://doc.rust-lang.org/guide-pointers.html#rc-and-arc).
guide](guide-pointers.html#rc-and-arc).
# Patterns
@ -5336,6 +5336,6 @@ you will have a firm grasp of basic Rust development. There's a whole lot more
out there, we've just covered the surface. There's tons of topics that you can
dig deeper into, and we've built specialized guides for many of them. To learn
more, dig into the [full documentation
index](http://doc.rust-lang.org/index.html).
index](index.html).
Happy hacking!

View file

@ -63,6 +63,7 @@ a guide that can help you out:
* [Macros](guide-macros.html)
* [Testing](guide-testing.html)
* [Rust's Runtime](guide-runtime.html)
* [Compiler Plugins](guide-plugin.html)
# Tools

View file

@ -13,6 +13,7 @@
[type: text] src/doc/guide-ffi.md $lang:doc/l10n/$lang/guide-ffi.md
[type: text] src/doc/guide-lifetimes.md $lang:doc/l10n/$lang/guide-lifetimes.md
[type: text] src/doc/guide-macros.md $lang:doc/l10n/$lang/guide-macros.md
[type: text] src/doc/guide-plugin.md $lang:doc/l10n/$lang/guide-plugin.md
[type: text] src/doc/guide-pointers.md $lang:doc/l10n/$lang/guide-pointers.md
[type: text] src/doc/guide-runtime.md $lang:doc/l10n/$lang/guide-runtime.md
[type: text] src/doc/guide-strings.md $lang:doc/l10n/$lang/guide-strings.md

View file

@ -598,6 +598,14 @@ names, and invoked through a consistent syntax: `name!(...)`. Examples include:
All of the above extensions are expressions with values.
Users of `rustc` can define new syntax extensions in two ways:
* [Compiler plugins](guide-plugin.html#syntax-extensions) can include arbitrary
Rust code that manipulates syntax trees at compile time.
* [Macros](guide-macros.html) define new syntax in a higher-level,
declarative way.
## Macros
```{.ebnf .gram}
@ -615,7 +623,7 @@ transcriber : '(' transcriber * ')' | '[' transcriber * ']'
User-defined syntax extensions are called "macros", and the `macro_rules`
syntax extension defines them. Currently, user-defined macros can expand to
expressions, statements, or items.
expressions, statements, items, or patterns.
(A `sep_token` is any token other than `*` and `+`. A `non_special_token` is
any token other than a delimiter or `$`.)
@ -1912,7 +1920,7 @@ type int8_t = i8;
- `main` - indicates that this function should be passed to the entry point,
rather than the function in the crate root named `main`.
- `plugin_registrar` - mark this function as the registration point for
compiler plugins, such as loadable syntax extensions.
[compiler plugins][plugin], such as loadable syntax extensions.
- `start` - indicates that this function should be used as the entry point,
overriding the "start" language item. See the "start" [language
item](#language-items) for more details.
@ -1972,8 +1980,8 @@ On `struct`s:
align fields.
- `phase` - on `extern crate` statements, allows specifying which "phase" of
compilation the crate should be loaded for. Currently, there are two
choices: `link` and `plugin`. `link` is the default. `plugin` will load the
crate at compile-time and use any syntax extensions or lints that the crate
choices: `link` and `plugin`. `link` is the default. `plugin` will [load the
crate at compile-time][plugin] and use any syntax extensions or lints that the crate
defines. They can both be specified, `#[phase(link, plugin)]` to use a crate
both at runtime and compiletime.
- `simd` - on certain tuple structs, derive the arithmetic operators, which
@ -2061,7 +2069,8 @@ For any lint check `C`:
* `warn(C)` warns about violations of `C` but continues compilation.
The lint checks supported by the compiler can be found via `rustc -W help`,
along with their default settings.
along with their default settings. [Compiler
plugins](guide-plugin.html#lint-plugins) can provide additional lint checks.
```{.ignore}
mod m1 {
@ -2490,7 +2499,7 @@ The currently implemented features of the reference compiler are:
considered unwholesome and in need of overhaul, and it is not clear
what they will look like moving forward.
* `plugin_registrar` - Indicates that a crate has compiler plugins that it
* `plugin_registrar` - Indicates that a crate has [compiler plugins][plugin] that it
wants to load. As with `phase`, the implementation is
in need of a overhaul, and it is not clear that plugins
defined using this will continue to work.
@ -4304,3 +4313,4 @@ Additional specific influences can be seen from the following languages:
* The block syntax of Ruby.
[ffi]: guide-ffi.html
[plugin]: guide-plugin.html

View file

@ -213,6 +213,10 @@ pre code {
color: inherit;
}
a > code {
color: #428BCA;
}
/* Code highlighting */
pre.rust .kw { color: #8959A8; }
pre.rust .kw-2, pre.rust .prelude-ty { color: #4271AE; }
@ -334,6 +338,11 @@ pre.rust { position: relative; }
transform: scaleX(-1);
}
.unstable-feature {
border: 2px solid red;
padding: 5px;
}
@media (min-width: 1170px) {
pre {
font-size: 15px;

View file

@ -53,8 +53,8 @@
* If you also need the plugin crate available at runtime, use
* `phase(plugin, link)`.
*
* See `src/test/auxiliary/macro_crate_test.rs` and `src/libfourcc`
* for examples of syntax extension plugins.
* See [the compiler plugin guide](../../guide-plugin.html)
* for more examples.
*/
pub use self::registry::Registry;

View file

@ -73,7 +73,7 @@ static KNOWN_FEATURES: &'static [(&'static str, Status)] = &[
("if_let", Active),
// if you change this list without updating src/doc/rust.md, cmr will be sad
// if you change this list without updating src/doc/reference.md, cmr will be sad
// A temporary feature gate used to enable parser extensions needed
// to bootstrap fix for #5723.

View file

@ -0,0 +1,70 @@
// Copyright 2014 The Rust Project Developers. See the COPYRIGHT
// file at the top-level directory of this distribution and at
// http://rust-lang.org/COPYRIGHT.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.
// force-host
#![crate_type="dylib"]
#![feature(plugin_registrar)]
extern crate syntax;
extern crate rustc;
use syntax::codemap::Span;
use syntax::parse::token::{IDENT, get_ident};
use syntax::ast::{TokenTree, TTTok};
use syntax::ext::base::{ExtCtxt, MacResult, DummyResult, MacExpr};
use syntax::ext::build::AstBuilder; // trait for expr_uint
use rustc::plugin::Registry;
// WARNING WARNING WARNING WARNING WARNING
// =======================================
//
// This code also appears in src/doc/guide-plugin.md. Please keep
// the two copies in sync! FIXME: have rustdoc read this file
fn expand_rn(cx: &mut ExtCtxt, sp: Span, args: &[TokenTree])
-> Box<MacResult + 'static> {
static NUMERALS: &'static [(&'static str, uint)] = &[
("M", 1000), ("CM", 900), ("D", 500), ("CD", 400),
("C", 100), ("XC", 90), ("L", 50), ("XL", 40),
("X", 10), ("IX", 9), ("V", 5), ("IV", 4),
("I", 1)];
let text = match args {
[TTTok(_, IDENT(s, _))] => get_ident(s).to_string(),
_ => {
cx.span_err(sp, "argument should be a single identifier");
return DummyResult::any(sp);
}
};
let mut text = text.as_slice();
let mut total = 0u;
while !text.is_empty() {
match NUMERALS.iter().find(|&&(rn, _)| text.starts_with(rn)) {
Some(&(rn, val)) => {
total += val;
text = text.slice_from(rn.len());
}
None => {
cx.span_err(sp, "invalid Roman numeral");
return DummyResult::any(sp);
}
}
}
MacExpr::new(cx.expr_uint(sp, total))
}
#[plugin_registrar]
pub fn plugin_registrar(reg: &mut Registry) {
reg.register_macro("rn", expand_rn);
}

View file

@ -0,0 +1,27 @@
// Copyright 2014 The Rust Project Developers. See the COPYRIGHT
// file at the top-level directory of this distribution and at
// http://rust-lang.org/COPYRIGHT.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.
// aux-build:roman_numerals.rs
// ignore-stage1
// ignore-android
#![feature(phase)]
#[phase(plugin, link)]
extern crate roman_numerals;
pub fn main() {
assert_eq!(rn!(MMXV), 2015);
assert_eq!(rn!(MCMXCIX), 1999);
assert_eq!(rn!(XXV), 25);
assert_eq!(rn!(MDCLXVI), 1666);
assert_eq!(rn!(MMMDCCCLXXXVIII), 3888);
assert_eq!(rn!(MMXIV), 2014);
}