squiid/rust
1
0
Fork 0
Code Issues Pull requests Projects Activity

auto merge of #11151 : sfackler/rust/ext-crate, r=alexcrichton

Browse source 
This is a first pass on support for procedural macros that aren't hardcoded into libsyntax. It is **not yet ready to merge** but I've opened a PR to have a chance to discuss some open questions and implementation issues.

Example
=======
Here's a silly example showing off the basics:

my_synext.rs
```rust
#[feature(managed_boxes, globs, macro_registrar, macro_rules)];

extern mod syntax;

use syntax::ast::{Name, token_tree};
use syntax::codemap::Span;
use syntax::ext::base::*;
use syntax::parse::token;

#[macro_export]
macro_rules! exported_macro (() => (2))

#[macro_registrar]
pub fn macro_registrar(register: |Name, SyntaxExtension|) {
    register(token::intern(&"make_a_1"),
        NormalTT(@SyntaxExpanderTT {
            expander: SyntaxExpanderTTExpanderWithoutContext(expand_make_a_1),
            span: None,
        } as @SyntaxExpanderTTTrait,
        None));
}

pub fn expand_make_a_1(cx: &mut ExtCtxt, sp: Span, tts: &[token_tree]) -> MacResult {
    if !tts.is_empty() {
        cx.span_fatal(sp, "make_a_1 takes no arguments");
    }
    MRExpr(quote_expr!(cx, 1i))
}
```

main.rs:
```rust
#[feature(phase)];

#[phase(syntax)]
extern mod my_synext;

fn main() {
    assert_eq!(1, make_a_1!());
    assert_eq!(2, exported_macro!());
}
```

Overview
=======
Crates that contain syntax extensions need to define a function with the following signature and annotation:
```rust
#[macro_registrar]
pub fn registrar(register: |ast::Name, ext::base::SyntaxExtension|) { ... }
```
that should call the `register` closure with each extension it defines. `macro_rules!` style macros can be tagged with `#[macro_export]` to be exported from the crate as well.

Crates that wish to use externally loadable syntax extensions load them by adding the `#[phase(syntax)]` attribute to an `extern mod`. All extensions registered by the specified crate are loaded with the same scoping rules as `macro_rules!` macros. If you want to use a crate both for syntax extensions and normal linkage, you can use `#[phase(syntax, link)]`.

Open questions
===========
* ~~Does the `macro_crate` syntax make sense? It wraps an entire `extern mod` declaration which looks a bit weird but is nice in the sense that the crate lookup logic can be identical between normal external crates and external macro crates. If the `extern mod` syntax, changes, this will get it for free, etc.~~ Changed to a `phase` attribute.
* ~~Is the magic name `macro_crate_registration` the right way to handle extension registration? It could alternatively be handled by a function annotated with `#[macro_registration]` I guess.~~ Switched to an attribute.
* The crate loading logic lives inside of librustc, which means that the syntax extension infrastructure can't directly access it. I've worked around this by passing a `CrateLoader` trait object from the driver to libsyntax that can call back into the crate loading logic. It should be possible to pull things apart enough that this isn't necessary anymore, but it will be an enormous refactoring project. I think we'll need to create a couple of new libraries: libsynext libmetadata/ty and libmiddle.
* Item decorator extensions can be loaded but the `deriving` decorator itself can't be extended so you'd need to do e.g. `#[deriving_MyTrait] #[deriving(Clone)]` instead of `#[deriving(MyTrait, Clone)]`. Is this something worth bothering with for now?

Remaining work
===========
- [x] ~~There is not yet support for rustdoc downloading and compiling referenced macro crates as it does for other referenced crates. This shouldn't be too hard I think.~~
- [x] ~~This is not testable at stage1 and sketchily testable at stages above that. The stage *n* rustc links against the stage *n-1* libsyntax and librustc. Unfortunately, crates in the test/auxiliary directory link against the stage *n* libstd, libextra, libsyntax, etc. This causes macro crates to fail to properly dynamically link into rustc since names end up being mangled slightly differently. In addition, when rustc is actually installed onto a system, there are actually do copies of libsyntax, libstd, etc: the ones that user code links against and a separate set from the previous stage that rustc itself uses. By this point in the bootstrap process, the two library versions *should probably* be binary compatible, but it doesn't seem like a sure thing. Fixing this is apparently hard, but necessary to properly cross compile as well and is being tracked in #11145.~~ The offending tests are ignored during `check-stage1-rpass` and `check-stage1-cfail`. When we get a snapshot that has this commit, I'll look into how feasible it'll be to get them working on stage1.
- [x] ~~`macro_rules!` style macros aren't being exported. Now that the crate loading infrastructure is there, this should just require serializing the AST of the macros into the crate metadata and yanking them out again, but I'm not very familiar with that part of the compiler.~~
- [x] ~~The `macro_crate_registration` function isn't type-checked when it's loaded. I poked around in the `csearch` infrastructure a bit but didn't find any super obvious ways of checking the type of an item with a certain name. Fixing this may also eliminate the need to `#[no_mangle]` the registration function.~~ Now that the registration function is identified by an attribute, typechecking this will be like typechecking other annotated functions.
- [x] ~~The dynamic libraries that are loaded are never unloaded. It shouldn't require too much work to tie the lifetime of the `DynamicLibrary` object to the `MapChain` that its extensions are loaded into.~~
- [x] ~~The compiler segfaults sometimes when loading external crates. The `DynamicLibrary` reference and code objects from that library are both put into the same hash table. When the table drops, due to the random ordering the library sometimes drops before the objects do. Once #11228 lands it'll be easy to fix this.~~
This commit is contained in:
bors 2014-01-16 16:36:53 -08:00
commit 80a3f453db
39 changed files with 872 additions and 204 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
src/compiletest/header.rs
View file

@ -90,10 +90,14 @@ pub fn is_test_ignored(config: &config, testfile: &Path) -> bool {
fn xfail_target(config: &config) -> ~str {
~"xfail-" + util::get_os(config.target)
}
fn xfail_stage(config: &config) -> ~str {
~"xfail-" + config.stage_id.split('-').next().unwrap()
}
let val = iter_header(testfile, |ln| {
if parse_name_directive(ln, "xfail-test") { false }
else if parse_name_directive(ln, xfail_target(config)) { false }
else if parse_name_directive(ln, xfail_stage(config)) { false }
else if config.mode == common::mode_pretty &&
parse_name_directive(ln, "xfail-pretty") { false }
else { true }

26
src/librustc/driver/driver.rs
View file

@ -20,6 +20,7 @@ use lib::llvm::{ContextRef, ModuleRef};
use metadata::common::LinkMeta;
use metadata::{creader, filesearch};
use metadata::cstore::CStore;
use metadata::creader::Loader;
use metadata;
use middle::{trans, freevars, kind, ty, typeck, lint, astencode, reachable};
use middle;
@ -41,6 +42,7 @@ use syntax::attr;
use syntax::attr::{AttrMetaMethods};
use syntax::codemap;
use syntax::diagnostic;
use syntax::ext::base::CrateLoader;
use syntax::parse;
use syntax::parse::token;
use syntax::print::{pp, pprust};
@ -163,6 +165,7 @@ pub fn phase_1_parse_input(sess: Session, cfg: ast::CrateConfig, input: &input)
/// standard library and prelude.
pub fn phase_2_configure_and_expand(sess: Session,
cfg: ast::CrateConfig,
loader: &mut CrateLoader,
mut crate: ast::Crate)
-> (ast::Crate, syntax::ast_map::Map) {
let time_passes = sess.time_passes();
@ -188,9 +191,14 @@ pub fn phase_2_configure_and_expand(sess: Session,
crate = time(time_passes, "configuration 1", crate, |crate|
front::config::strip_unconfigured_items(crate));
crate = time(time_passes, "expansion", crate, |crate|
syntax::ext::expand::expand_crate(sess.parse_sess, cfg.clone(),
crate));
crate = time(time_passes, "expansion", crate, |crate| {
syntax::ext::expand::expand_crate(sess.parse_sess,
loader,
cfg.clone(),
crate)
});
// dump the syntax-time crates
sess.cstore.reset();
// strip again, in case expansion added anything with a #[cfg].
crate = time(time_passes, "configuration 2", crate, |crate|
@ -248,6 +256,11 @@ pub fn phase_3_run_analysis_passes(sess: Session,
time(time_passes, "looking for entry point", (),
|_| middle::entry::find_entry_point(sess, crate, ast_map));
sess.macro_registrar_fn.with_mut(|r| *r =
time(time_passes, "looking for macro registrar", (), |_|
syntax::ext::registrar::find_macro_registrar(
sess.span_diagnostic, crate)));
let freevars = time(time_passes, "freevar finding", (), |_|
freevars::annotate_freevars(def_map, crate));
@ -491,7 +504,8 @@ pub fn compile_input(sess: Session, cfg: ast::CrateConfig, input: &input,
let (expanded_crate, ast_map) = {
let crate = phase_1_parse_input(sess, cfg.clone(), input);
if stop_after_phase_1(sess) { return; }
phase_2_configure_and_expand(sess, cfg, crate)
let loader = &mut Loader::new(sess);
phase_2_configure_and_expand(sess, cfg, loader, crate)
};
let outputs = build_output_filenames(input, outdir, output,
expanded_crate.attrs, sess);
@ -579,7 +593,8 @@ pub fn pretty_print_input(sess: Session,
let (crate, ast_map, is_expanded) = match ppm {
PpmExpanded | PpmExpandedIdentified | PpmTyped => {
let (crate, ast_map) = phase_2_configure_and_expand(sess, cfg, crate);
let loader = &mut Loader::new(sess);
let (crate, ast_map) = phase_2_configure_and_expand(sess, cfg, loader, crate);
(crate, Some(ast_map), true)
}
_ => (crate, None, false)
@ -912,6 +927,7 @@ pub fn build_session_(sopts: @session::options,
// For a library crate, this is always none
entry_fn: RefCell::new(None),
entry_type: Cell::new(None),
macro_registrar_fn: RefCell::new(None),
span_diagnostic: span_diagnostic_handler,
filesearch: filesearch,
building_library: Cell::new(false),

1
src/librustc/driver/session.rs
View file

@ -210,6 +210,7 @@ pub struct Session_ {
entry_fn: RefCell<Option<(NodeId, codemap::Span)>>,
entry_type: Cell<Option<EntryFnType>>,
span_diagnostic: @diagnostic::SpanHandler,
macro_registrar_fn: RefCell<Option<ast::DefId>>,
filesearch: @filesearch::FileSearch,
building_library: Cell<bool>,
working_dir: Path,

20
src/librustc/front/feature_gate.rs
View file

@ -43,6 +43,8 @@ static KNOWN_FEATURES: &'static [(&'static str, Status)] = &[
("non_ascii_idents", Active),
("thread_local", Active),
("link_args", Active),
("phase", Active),
("macro_registrar", Active),
// These are used to test this portion of the compiler, they don't actually
// mean anything
@ -114,7 +116,15 @@ impl Visitor<()> for Context {
}
}
}
_ => {}
ast::ViewItemExternMod(..) => {
for attr in i.attrs.iter() {
if "phase" == attr.name() {
self.gate_feature("phase", attr.span,
"compile time crate loading is \
experimental and possibly buggy");
}
}
}
}
visit::walk_view_item(self, i, ())
}
@ -151,6 +161,14 @@ impl Visitor<()> for Context {
}
}
ast::ItemFn(..) => {
if attr::contains_name(i.attrs, "macro_registrar") {
self.gate_feature("macro_registrar", i.span,
"cross-crate macro exports are \
experimental and possibly buggy");
}
}
_ => {}
}

14
src/librustc/front/test.rs
View file

@ -14,6 +14,7 @@
use driver::session;
use front::config;
use front::std_inject::with_version;
use metadata::creader::Loader;
use std::cell::RefCell;
use std::vec;
@ -38,10 +39,10 @@ struct Test {
should_fail: bool
}
struct TestCtxt {
struct TestCtxt<'a> {
sess: session::Session,
path: RefCell<~[ast::Ident]>,
ext_cx: ExtCtxt,
ext_cx: ExtCtxt<'a>,
testfns: RefCell<~[Test]>,
is_extra: bool,
config: ast::CrateConfig,
@ -63,11 +64,11 @@ pub fn modify_for_testing(sess: session::Session,
}
}
struct TestHarnessGenerator {
cx: TestCtxt,
struct TestHarnessGenerator<'a> {
cx: TestCtxt<'a>,
}
impl fold::Folder for TestHarnessGenerator {
impl<'a> fold::Folder for TestHarnessGenerator<'a> {
fn fold_crate(&mut self, c: ast::Crate) -> ast::Crate {
let folded = fold::noop_fold_crate(c, self);
@ -155,9 +156,10 @@ impl fold::Folder for TestHarnessGenerator {
fn generate_test_harness(sess: session::Session, crate: ast::Crate)
-> ast::Crate {
let loader = &mut Loader::new(sess);
let mut cx: TestCtxt = TestCtxt {
sess: sess,
ext_cx: ExtCtxt::new(sess.parse_sess, sess.opts.cfg.clone()),
ext_cx: ExtCtxt::new(sess.parse_sess, sess.opts.cfg.clone(), loader),
path: RefCell::new(~[]),
testfns: RefCell::new(~[]),
is_extra: is_extra(&crate),

4
src/librustc/metadata/common.rs
View file

@ -204,6 +204,10 @@ pub static tag_native_libraries_lib: uint = 0x104;
pub static tag_native_libraries_name: uint = 0x105;
pub static tag_native_libraries_kind: uint = 0x106;
pub static tag_macro_registrar_fn: uint = 0x110;
pub static tag_exported_macros: uint = 0x111;
pub static tag_macro_def: uint = 0x112;
#[deriving(Clone)]
pub struct LinkMeta {
crateid: CrateId,

135
src/librustc/metadata/creader.rs
View file

@ -10,10 +10,13 @@
//! Validates all used crates and extern libraries and loads their metadata
use driver::{driver, session};
use driver::session::Session;
use metadata::csearch;
use metadata::cstore;
use metadata::decoder;
use metadata::loader;
use metadata::loader::Os;
use std::cell::RefCell;
use std::hashmap::HashMap;
@ -23,6 +26,7 @@ use syntax::attr;
use syntax::attr::AttrMetaMethods;
use syntax::codemap::{Span, DUMMY_SP};
use syntax::diagnostic::SpanHandler;
use syntax::ext::base::{CrateLoader, MacroCrate};
use syntax::parse::token;
use syntax::parse::token::IdentInterner;
use syntax::crateid::CrateId;
@ -131,37 +135,65 @@ fn visit_crate(e: &Env, c: &ast::Crate) {
}
fn visit_view_item(e: &mut Env, i: &ast::ViewItem) {
let should_load = i.attrs.iter().all(|attr| {
"phase" != attr.name() ||
attr.meta_item_list().map_or(false, |phases| {
attr::contains_name(phases, "link")
})
});
if !should_load {
return;
}
match extract_crate_info(i) {
Some(info) => {
let cnum = resolve_crate(e, info.ident, info.name, info.version,
@"", i.span);
e.sess.cstore.add_extern_mod_stmt_cnum(info.id, cnum);
}
None => ()
}
}
struct CrateInfo {
ident: @str,
name: @str,
version: @str,
id: ast::NodeId,
}
fn extract_crate_info(i: &ast::ViewItem) -> Option<CrateInfo> {
match i.node {
ast::ViewItemExternMod(ident, path_opt, id) => {
let ident = token::ident_to_str(&ident);
debug!("resolving extern mod stmt. ident: {:?} path_opt: {:?}",
ident, path_opt);
let (name, version) = match path_opt {
Some((path_str, _)) => {
let crateid: Option<CrateId> = from_str(path_str);
match crateid {
None => (@"", @""),
Some(crateid) => {
let version = match crateid.version {
None => @"",
Some(ref ver) => ver.to_managed(),
};
(crateid.name.to_managed(), version)
}
}
}
None => (ident, @""),
};
let cnum = resolve_crate(e,
ident,
name,
version,
@"",
i.span);
e.sess.cstore.add_extern_mod_stmt_cnum(id, cnum);
}
_ => ()
}
ast::ViewItemExternMod(ident, path_opt, id) => {
let ident = token::ident_to_str(&ident);
debug!("resolving extern mod stmt. ident: {:?} path_opt: {:?}",
ident, path_opt);
let (name, version) = match path_opt {
Some((path_str, _)) => {
let crateid: Option<CrateId> = from_str(path_str);
match crateid {
None => (@"", @""),
Some(crateid) => {
let version = match crateid.version {
None => @"",
Some(ref ver) => ver.to_managed(),
};
(crateid.name.to_managed(), version)
}
}
}
None => (ident, @""),
};
Some(CrateInfo {
ident: ident,
name: name,
version: version,
id: id,
})
}
_ => None
}
}
fn visit_item(e: &Env, i: &ast::Item) {
@ -355,3 +387,46 @@ fn resolve_crate_deps(e: &mut Env, cdata: &[u8]) -> cstore::cnum_map {
}
return @RefCell::new(cnum_map);
}
pub struct Loader {
priv env: Env,
}
impl Loader {
pub fn new(sess: Session) -> Loader {
let os = driver::get_os(driver::host_triple()).unwrap();
let os = session::sess_os_to_meta_os(os);
Loader {
env: Env {
sess: sess,
os: os,
crate_cache: @RefCell::new(~[]),
next_crate_num: 1,
intr: token::get_ident_interner(),
}
}
}
}
impl CrateLoader for Loader {
fn load_crate(&mut self, crate: &ast::ViewItem) -> MacroCrate {
let info = extract_crate_info(crate).unwrap();
let cnum = resolve_crate(&mut self.env, info.ident, info.name,
info.version, @"", crate.span);
let library = self.env.sess.cstore.get_used_crate_source(cnum).unwrap();
MacroCrate {
lib: library.dylib,
cnum: cnum
}
}
fn get_exported_macros(&mut self, cnum: ast::CrateNum) -> ~[@ast::Item] {
csearch::get_exported_macros(self.env.sess.cstore, cnum)
}
fn get_registrar_symbol(&mut self, cnum: ast::CrateNum) -> Option<~str> {
let cstore = self.env.sess.cstore;
csearch::get_macro_registrar_fn(cstore, cnum)
.map(|did| csearch::get_symbol(cstore, did))
}
}

13
src/librustc/metadata/csearch.rs
View file

@ -301,3 +301,16 @@ pub fn get_trait_of_method(cstore: @cstore::CStore,
decoder::get_trait_of_method(cdata, def_id.node, tcx)
}
pub fn get_macro_registrar_fn(cstore: @cstore::CStore,
crate_num: ast::CrateNum)
-> Option<ast::DefId> {
let cdata = cstore.get_crate_data(crate_num);
decoder::get_macro_registrar_fn(cdata)
}
pub fn get_exported_macros(cstore: @cstore::CStore,
crate_num: ast::CrateNum)
-> ~[@ast::Item] {
let cdata = cstore.get_crate_data(crate_num);
decoder::get_exported_macros(cdata)
}

17
src/librustc/metadata/cstore.rs
View file

@ -53,7 +53,7 @@ pub enum NativeLibaryKind {
// Where a crate came from on the local filesystem. One of these two options
// must be non-None.
#[deriving(Eq)]
#[deriving(Eq, Clone)]
pub struct CrateSource {
dylib: Option<Path>,
rlib: Option<Path>,
@ -123,6 +123,21 @@ impl CStore {
}
}
pub fn get_used_crate_source(&self, cnum: ast::CrateNum)
-> Option<CrateSource> {
let mut used_crate_sources = self.used_crate_sources.borrow_mut();
used_crate_sources.get().iter().find(|source| source.cnum == cnum)
.map(|source| source.clone())
}
pub fn reset(&self) {
self.metas.with_mut(|s| s.clear());
self.extern_mod_crate_map.with_mut(|s| s.clear());
self.used_crate_sources.with_mut(|s| s.clear());
self.used_libraries.with_mut(|s| s.clear());
self.used_link_args.with_mut(|s| s.clear());
}
pub fn get_used_crates(&self, prefer: LinkagePreference)
-> ~[(ast::CrateNum, Option<Path>)] {
let used_crate_sources = self.used_crate_sources.borrow();

17
src/librustc/metadata/decoder.rs
View file

@ -23,6 +23,7 @@ use metadata::tydecode::{parse_ty_data, parse_def_id,
use middle::ty::{ImplContainer, TraitContainer};
use middle::ty;
use middle::typeck;
use middle::astencode;
use middle::astencode::vtable_decoder_helpers;
use std::at_vec;
@ -1275,3 +1276,19 @@ pub fn get_native_libraries(cdata: Cmd) -> ~[(cstore::NativeLibaryKind, ~str)] {
});
return result;
}
pub fn get_macro_registrar_fn(cdata: Cmd) -> Option<ast::DefId> {
reader::maybe_get_doc(reader::Doc(cdata.data()), tag_macro_registrar_fn)
.map(|doc| item_def_id(doc, cdata))
}
pub fn get_exported_macros(cdata: Cmd) -> ~[@ast::Item] {
let macros = reader::get_doc(reader::Doc(cdata.data()),
tag_exported_macros);
let mut result = ~[];
reader::tagged_docs(macros, tag_macro_def, |macro_doc| {
result.push(astencode::decode_exported_macro(macro_doc));
true
});
result
}

86
src/librustc/metadata/encoder.rs
View file

@ -79,6 +79,8 @@ struct Stats {
dep_bytes: Cell<u64>,
lang_item_bytes: Cell<u64>,
native_lib_bytes: Cell<u64>,
macro_registrar_fn_bytes: Cell<u64>,
macro_defs_bytes: Cell<u64>,
impl_bytes: Cell<u64>,
misc_bytes: Cell<u64>,
item_bytes: Cell<u64>,
@ -1287,7 +1289,9 @@ fn encode_info_for_item(ecx: &EncodeContext,
// Encode inherent implementations for this trait.
encode_inherent_implementations(ecx, ebml_w, def_id);
}
ItemMac(..) => fail!("item macros unimplemented")
ItemMac(..) => {
// macros are encoded separately
}
}
}
@ -1691,6 +1695,50 @@ fn encode_native_libraries(ecx: &EncodeContext, ebml_w: &mut writer::Encoder) {
ebml_w.end_tag();
}
fn encode_macro_registrar_fn(ecx: &EncodeContext, ebml_w: &mut writer::Encoder) {
let ptr = ecx.tcx.sess.macro_registrar_fn.borrow();
match *ptr.get() {
Some(did) => {
ebml_w.start_tag(tag_macro_registrar_fn);
encode_def_id(ebml_w, did);
ebml_w.end_tag();
}
None => {}
}
}
struct MacroDefVisitor<'a, 'b> {
ecx: &'a EncodeContext<'a>,
ebml_w: &'a mut writer::Encoder<'b>
}
impl<'a, 'b> Visitor<()> for MacroDefVisitor<'a, 'b> {
fn visit_item(&mut self, item: &Item, _: ()) {
match item.node {
ItemMac(..) => {
self.ebml_w.start_tag(tag_macro_def);
astencode::encode_exported_macro(self.ebml_w, item);
self.ebml_w.end_tag();
}
_ => {}
}
}
}
fn encode_macro_defs(ecx: &EncodeContext,
crate: &Crate,
ebml_w: &mut writer::Encoder) {
ebml_w.start_tag(tag_exported_macros);
{
let mut visitor = MacroDefVisitor {
ecx: ecx,
ebml_w: ebml_w,
};
visit::walk_crate(&mut visitor, crate, ());
}
ebml_w.end_tag();
}
struct ImplVisitor<'a,'b> {
ecx: &'a EncodeContext<'a>,
ebml_w: &'a mut writer::Encoder<'b>,
@ -1815,6 +1863,8 @@ fn encode_metadata_inner(wr: &mut MemWriter, parms: EncodeParams, crate: &Crate)
dep_bytes: Cell::new(0),
lang_item_bytes: Cell::new(0),
native_lib_bytes: Cell::new(0),
macro_registrar_fn_bytes: Cell::new(0),
macro_defs_bytes: Cell::new(0),
impl_bytes: Cell::new(0),
misc_bytes: Cell::new(0),
item_bytes: Cell::new(0),
@ -1873,6 +1923,16 @@ fn encode_metadata_inner(wr: &mut MemWriter, parms: EncodeParams, crate: &Crate)
encode_native_libraries(&ecx, &mut ebml_w);
ecx.stats.native_lib_bytes.set(ebml_w.writer.tell() - i);
// Encode the macro registrar function
i = ebml_w.writer.tell();
encode_macro_registrar_fn(&ecx, &mut ebml_w);
ecx.stats.macro_registrar_fn_bytes.set(ebml_w.writer.tell() - i);
// Encode macro definitions
i = ebml_w.writer.tell();
encode_macro_defs(&ecx, crate, &mut ebml_w);
ecx.stats.macro_defs_bytes.set(ebml_w.writer.tell() - i);
// Encode the def IDs of impls, for coherence checking.
i = ebml_w.writer.tell();
encode_impls(&ecx, crate, &mut ebml_w);
@ -1905,17 +1965,19 @@ fn encode_metadata_inner(wr: &mut MemWriter, parms: EncodeParams, crate: &Crate)
}
println!("metadata stats:");
println!(" inline bytes: {}", ecx.stats.inline_bytes.get());
println!(" attribute bytes: {}", ecx.stats.attr_bytes.get());
println!(" dep bytes: {}", ecx.stats.dep_bytes.get());
println!(" lang item bytes: {}", ecx.stats.lang_item_bytes.get());
println!(" native bytes: {}", ecx.stats.native_lib_bytes.get());
println!(" impl bytes: {}", ecx.stats.impl_bytes.get());
println!(" misc bytes: {}", ecx.stats.misc_bytes.get());
println!(" item bytes: {}", ecx.stats.item_bytes.get());
println!(" index bytes: {}", ecx.stats.index_bytes.get());
println!(" zero bytes: {}", ecx.stats.zero_bytes.get());
println!(" total bytes: {}", ecx.stats.total_bytes.get());
println!(" inline bytes: {}", ecx.stats.inline_bytes.get());
println!(" attribute bytes: {}", ecx.stats.attr_bytes.get());
println!(" dep bytes: {}", ecx.stats.dep_bytes.get());
println!(" lang item bytes: {}", ecx.stats.lang_item_bytes.get());
println!(" native bytes: {}", ecx.stats.native_lib_bytes.get());
println!("macro registrar bytes: {}", ecx.stats.macro_registrar_fn_bytes.get());
println!(" macro def bytes: {}", ecx.stats.macro_defs_bytes.get());
println!(" impl bytes: {}", ecx.stats.impl_bytes.get());
println!(" misc bytes: {}", ecx.stats.misc_bytes.get());
println!(" item bytes: {}", ecx.stats.item_bytes.get());
println!(" index bytes: {}", ecx.stats.index_bytes.get());
println!(" zero bytes: {}", ecx.stats.zero_bytes.get());
println!(" total bytes: {}", ecx.stats.total_bytes.get());
}
}

14
src/librustc/middle/astencode.rs
View file

@ -107,6 +107,13 @@ pub fn encode_inlined_item(ecx: &e::EncodeContext,
ebml_w.writer.tell());
}
pub fn encode_exported_macro(ebml_w: &mut writer::Encoder, i: &ast::Item) {
match i.node {
ast::ItemMac(..) => encode_ast(ebml_w, ast::IIItem(@i.clone())),
_ => fail!("expected a macro")
}
}
pub fn decode_inlined_item(cdata: @cstore::crate_metadata,
tcx: ty::ctxt,
maps: Maps,
@ -159,6 +166,13 @@ pub fn decode_inlined_item(cdata: @cstore::crate_metadata,
}
}
pub fn decode_exported_macro(par_doc: ebml::Doc) -> @ast::Item {
match decode_ast(par_doc) {
ast::IIItem(item) => item,
_ => fail!("expected item")
}
}
// ______________________________________________________________________
// Enumerating the IDs which appear in an AST

7
src/librustc/middle/lint.rs
View file

@ -924,15 +924,16 @@ static other_attrs: &'static [&'static str] = &[
"allow", "deny", "forbid", "warn", // lint options
"deprecated", "experimental", "unstable", "stable", "locked", "frozen", //item stability
"crate_map", "cfg", "doc", "export_name", "link_section", "no_freeze",
"no_mangle", "no_send", "static_assert", "unsafe_no_drop_flag",
"packed", "simd", "repr", "deriving", "unsafe_destructor", "link",
"no_mangle", "no_send", "static_assert", "unsafe_no_drop_flag", "packed",
"simd", "repr", "deriving", "unsafe_destructor", "link", "phase",
"macro_export",
//mod-level
"path", "link_name", "link_args", "nolink", "macro_escape", "no_implicit_prelude",
// fn-level
"test", "bench", "should_fail", "ignore", "inline", "lang", "main", "start",
"no_split_stack", "cold",
"no_split_stack", "cold", "macro_registrar",
// internal attribute: bypass privacy inside items
"!resolve_unexported",

11
src/librustc/middle/resolve.rs
View file

@ -1400,10 +1400,7 @@ impl Resolver {
name_bindings.define_type(DefTrait(def_id), sp, is_public);
new_parent
}
ItemMac(..) => {
fail!("item macros unimplemented")
}
ItemMac(..) => parent
}
}
@ -3804,9 +3801,9 @@ impl Resolver {
});
}
ItemMac(..) => {
fail!("item macros unimplemented")
}
ItemMac(..) => {
// do nothing, these are just around to be encoded
}
}
}

5
src/librustc/middle/typeck/collect.rs
View file

@ -560,7 +560,7 @@ pub fn convert(ccx: &CrateCtxt, it: &ast::Item) {
debug!("convert: item {} with id {}", tcx.sess.str_of(it.ident), it.id);
match it.node {
// These don't define types.
ast::ItemForeignMod(_) | ast::ItemMod(_) => {}
ast::ItemForeignMod(_) | ast::ItemMod(_) | ast::ItemMac(_) => {}
ast::ItemEnum(ref enum_definition, ref generics) => {
ensure_no_ty_param_bounds(ccx, it.span, generics, "enumeration");
let tpt = ty_of_item(ccx, it);
@ -913,8 +913,7 @@ pub fn ty_of_item(ccx: &CrateCtxt, it: &ast::Item)
return tpt;
}
ast::ItemImpl(..) | ast::ItemMod(_) |
ast::ItemForeignMod(_) => fail!(),
ast::ItemMac(..) => fail!("item macros unimplemented")
ast::ItemForeignMod(_) | ast::ItemMac(_) => fail!(),
}
}

4
src/librustdoc/core.rs
View file

@ -10,6 +10,7 @@
use rustc;
use rustc::{driver, middle};
use rustc::metadata::creader::Loader;
use rustc::middle::privacy;
use syntax::ast;
@ -74,7 +75,8 @@ fn get_ast_and_resolve(cpath: &Path,
}
let crate = phase_1_parse_input(sess, cfg.clone(), &input);
let (crate, ast_map) = phase_2_configure_and_expand(sess, cfg, crate);
let loader = &mut Loader::new(sess);
let (crate, ast_map) = phase_2_configure_and_expand(sess, cfg, loader, crate);
let driver::driver::CrateAnalysis {
exported_items, public_items, ty_cx, ..
} = phase_3_run_analysis_passes(sess, &crate, ast_map);

4
src/librustdoc/test.rs
View file

@ -20,6 +20,7 @@ use extra::getopts;
use extra::test;
use rustc::driver::driver;
use rustc::driver::session;
use rustc::metadata::creader::Loader;
use syntax::diagnostic;
use syntax::parse;
@ -58,7 +59,8 @@ pub fn run(input: &str, matches: &getopts::Matches) -> int {
let cfg = driver::build_configuration(sess);
let crate = driver::phase_1_parse_input(sess, cfg.clone(), &input);
let (crate, _) = driver::phase_2_configure_and_expand(sess, cfg, crate);
let loader = &mut Loader::new(sess);
let (crate, _) = driver::phase_2_configure_and_expand(sess, cfg, loader, crate);
let ctx = @core::DocContext {
crate: crate,

7
src/librustpkg/lib.rs
View file

@ -28,6 +28,7 @@ pub use std::path::Path;
use extra::workcache;
use rustc::driver::{driver, session};
use rustc::metadata::creader::Loader;
use rustc::metadata::filesearch;
use rustc::metadata::filesearch::rust_path;
use rustc::util::sha2;
@ -118,7 +119,11 @@ impl<'a> PkgScript<'a> {
@diagnostic::Emitter);
let cfg = driver::build_configuration(sess);
let crate = driver::phase_1_parse_input(sess, cfg.clone(), &input);
let crate_and_map = driver::phase_2_configure_and_expand(sess, cfg.clone(), crate);
let loader = &mut Loader::new(sess);
let crate_and_map = driver::phase_2_configure_and_expand(sess,
cfg.clone(),
loader,
crate);
let work_dir = build_pkg_id_in_workspace(id, workspace);
debug!("Returning package script with id {}", id.to_str());

118
src/librustpkg/util.rs
View file

@ -16,11 +16,13 @@ use std::os;
use std::io;
use std::io::fs;
use extra::workcache;
use rustc::metadata::creader::Loader;
use rustc::driver::{driver, session};
use extra::getopts::groups::getopts;
use syntax;
use syntax::codemap::{DUMMY_SP, Spanned};
use syntax::ext::base::ExtCtxt;
use syntax::ext::base;
use syntax::ext::base::{ExtCtxt, MacroCrate};
use syntax::{ast, attr, codemap, diagnostic, fold, visit};
use syntax::attr::AttrMetaMethods;
use syntax::fold::Folder;
@ -63,9 +65,9 @@ struct ListenerFn {
path: ~[ast::Ident]
}
struct ReadyCtx {
struct ReadyCtx<'a> {
sess: session::Session,
ext_cx: ExtCtxt,
ext_cx: ExtCtxt<'a>,
path: ~[ast::Ident],
fns: ~[ListenerFn]
}
@ -130,7 +132,7 @@ fn fold_item(item: @ast::Item, fold: &mut CrateSetup)
}
struct CrateSetup<'a> {
ctx: &'a mut ReadyCtx,
ctx: &'a mut ReadyCtx<'a>,
}
impl<'a> fold::Folder for CrateSetup<'a> {
@ -145,9 +147,10 @@ impl<'a> fold::Folder for CrateSetup<'a> {
/// Generate/filter main function, add the list of commands, etc.
pub fn ready_crate(sess: session::Session,
crate: ast::Crate) -> ast::Crate {
let loader = &mut Loader::new(sess);
let mut ctx = ReadyCtx {
sess: sess,
ext_cx: ExtCtxt::new(sess.parse_sess, sess.opts.cfg.clone()),
ext_cx: ExtCtxt::new(sess.parse_sess, sess.opts.cfg.clone(), loader),
path: ~[],
fns: ~[]
};
@ -269,23 +272,41 @@ pub fn compile_input(context: &BuildContext,
// `extern mod` directives.
let cfg = driver::build_configuration(sess);
let crate = driver::phase_1_parse_input(sess, cfg.clone(), &input);
let (mut crate, ast_map) = driver::phase_2_configure_and_expand(sess, cfg.clone(), crate);
debug!("About to call find_and_install_dependencies...");
find_and_install_dependencies(context, pkg_id, in_file, sess, exec, &crate, deps,
|p| {
debug!("a dependency: {}", p.display());
let mut addl_lib_search_paths =
addl_lib_search_paths.borrow_mut();
let addl_lib_search_paths =
addl_lib_search_paths.get();
let mut addl_lib_search_paths =
addl_lib_search_paths.borrow_mut();
// Pass the directory containing a dependency
// as an additional lib search path
addl_lib_search_paths.get().insert(p);
});
let (mut crate, ast_map) = {
let installer = CrateInstaller {
context: context,
parent: pkg_id,
parent_crate: in_file,
sess: sess,
exec: exec,
deps: deps,
save: |p| {
debug!("a dependency: {}", p.display());
let mut addl_lib_search_paths =
addl_lib_search_paths.borrow_mut();
let addl_lib_search_paths =
addl_lib_search_paths.get();
let mut addl_lib_search_paths =
addl_lib_search_paths.borrow_mut();
// Pass the directory containing a dependency
// as an additional lib search path
addl_lib_search_paths.get().insert(p);
},
};
let mut loader = CrateLoader {
installer: installer,
loader: Loader::new(sess),
};
let (crate, ast_map) = driver::phase_2_configure_and_expand(sess,
cfg.clone(),
&mut loader,
crate);
let CrateLoader { mut installer, .. } = loader;
debug!("About to call find_and_install_dependencies...");
find_and_install_dependencies(&mut installer, &crate);
(crate, ast_map)
};
// Inject the crate_id attribute so we get the right package name and version
if !attr::contains_name(crate.attrs, "crate_id") {
@ -430,19 +451,18 @@ pub fn compile_crate(ctxt: &BuildContext,
compile_input(ctxt, exec, pkg_id, crate, workspace, deps, flags, cfgs, opt, what)
}
struct ViewItemVisitor<'a> {
struct CrateInstaller<'a> {
context: &'a BuildContext,
parent: &'a CrateId,
parent_crate: &'a Path,
sess: session::Session,
exec: &'a mut workcache::Exec,
c: &'a ast::Crate,
save: 'a |Path|,
deps: &'a mut DepMap
}
impl<'a> Visitor<()> for ViewItemVisitor<'a> {
fn visit_view_item(&mut self, vi: &ast::ViewItem, env: ()) {
impl<'a> CrateInstaller<'a> {
fn install_crate(&mut self, vi: &ast::ViewItem) {
use conditions::nonexistent_package::cond;
match vi.node {
@ -585,33 +605,43 @@ impl<'a> Visitor<()> for ViewItemVisitor<'a> {
// Ignore `use`s
_ => ()
}
}
}
impl<'a> Visitor<()> for CrateInstaller<'a> {
fn visit_view_item(&mut self, vi: &ast::ViewItem, env: ()) {
self.install_crate(vi);
visit::walk_view_item(self, vi, env)
}
}
struct CrateLoader<'a> {
installer: CrateInstaller<'a>,
loader: Loader,
}
impl<'a> base::CrateLoader for CrateLoader<'a> {
fn load_crate(&mut self, crate: &ast::ViewItem) -> MacroCrate {
self.installer.install_crate(crate);
self.loader.load_crate(crate)
}
fn get_exported_macros(&mut self, cnum: ast::CrateNum) -> ~[@ast::Item] {
self.loader.get_exported_macros(cnum)
}
fn get_registrar_symbol(&mut self, cnum: ast::CrateNum) -> Option<~str> {
self.loader.get_registrar_symbol(cnum)
}
}
/// Collect all `extern mod` directives in `c`, then
/// try to install their targets, failing if any target
/// can't be found.
pub fn find_and_install_dependencies(context: &BuildContext,
parent: &CrateId,
parent_crate: &Path,
sess: session::Session,
exec: &mut workcache::Exec,
c: &ast::Crate,
deps: &mut DepMap,
save: |Path|) {
pub fn find_and_install_dependencies(installer: &mut CrateInstaller,
c: &ast::Crate) {
debug!("In find_and_install_dependencies...");
let mut visitor = ViewItemVisitor {
context: context,
parent: parent,
parent_crate: parent_crate,
sess: sess,
exec: exec,
c: c,
save: save,
deps: deps
};
visit::walk_crate(&mut visitor, c, ())
visit::walk_crate(installer, c, ())
}
pub fn mk_string_lit(s: @str) -> ast::Lit {

111
src/libsyntax/ext/base.rs
View file

@ -20,6 +20,7 @@ use parse::token::{ident_to_str, intern, str_to_ident};
use util::small_vector::SmallVector;
use std::hashmap::HashMap;
use std::unstable::dynamic_lib::DynamicLibrary;
// new-style macro! tt code:
//
@ -120,6 +121,9 @@ pub type SyntaxExpanderTTItemFun =
pub type SyntaxExpanderTTItemFunNoCtxt =
fn(&mut ExtCtxt, Span, ast::Ident, ~[ast::TokenTree]) -> MacResult;
pub type MacroCrateRegistrationFun =
extern "Rust" fn(|ast::Name, SyntaxExtension|);
pub trait AnyMacro {
fn make_expr(&self) -> @ast::Expr;
fn make_items(&self) -> SmallVector<@ast::Item>;
@ -151,24 +155,21 @@ pub enum SyntaxExtension {
IdentTT(~SyntaxExpanderTTItemTrait:'static, Option<Span>),
}
// The SyntaxEnv is the environment that's threaded through the expansion
// of macros. It contains bindings for macros, and also a special binding
// for " block" (not a legal identifier) that maps to a BlockInfo
pub type SyntaxEnv = MapChain<Name, SyntaxExtension>;
pub struct BlockInfo {
// should macros escape from this scope?
macros_escape : bool,
// what are the pending renames?
pending_renames : RenameList
pending_renames : RenameList,
// references for crates loaded in this scope
macro_crates: ~[DynamicLibrary],
}
impl BlockInfo {
pub fn new() -> BlockInfo {
BlockInfo {
macros_escape: false,
pending_renames: ~[]
pending_renames: ~[],
macro_crates: ~[],
}
}
}
@ -189,7 +190,7 @@ pub fn syntax_expander_table() -> SyntaxEnv {
None)
}
let mut syntax_expanders = MapChain::new();
let mut syntax_expanders = SyntaxEnv::new();
syntax_expanders.insert(intern(&"macro_rules"),
IdentTT(~SyntaxExpanderTTItem {
expander: SyntaxExpanderTTItemExpanderWithContext(
@ -280,25 +281,38 @@ pub fn syntax_expander_table() -> SyntaxEnv {
syntax_expanders
}
pub struct MacroCrate {
lib: Option<Path>,
cnum: ast::CrateNum,
}
pub trait CrateLoader {
fn load_crate(&mut self, crate: &ast::ViewItem) -> MacroCrate;
fn get_exported_macros(&mut self, crate_num: ast::CrateNum) -> ~[@ast::Item];
fn get_registrar_symbol(&mut self, crate_num: ast::CrateNum) -> Option<~str>;
}
// One of these is made during expansion and incrementally updated as we go;
// when a macro expansion occurs, the resulting nodes have the backtrace()
// -> expn_info of their expansion context stored into their span.
pub struct ExtCtxt {
pub struct ExtCtxt<'a> {
parse_sess: @parse::ParseSess,
cfg: ast::CrateConfig,
backtrace: Option<@ExpnInfo>,
loader: &'a mut CrateLoader,
mod_path: ~[ast::Ident],
trace_mac: bool
}
impl ExtCtxt {
pub fn new(parse_sess: @parse::ParseSess, cfg: ast::CrateConfig)
-> ExtCtxt {
impl<'a> ExtCtxt<'a> {
pub fn new<'a>(parse_sess: @parse::ParseSess, cfg: ast::CrateConfig,
loader: &'a mut CrateLoader) -> ExtCtxt<'a> {
ExtCtxt {
parse_sess: parse_sess,
cfg: cfg,
backtrace: None,
loader: loader,
mod_path: ~[],
trace_mac: false
}
@ -456,20 +470,27 @@ pub fn get_exprs_from_tts(cx: &ExtCtxt,
// able to refer to a macro that was added to an enclosing
// scope lexically later than the deeper scope.
// Only generic to make it easy to test
struct MapChainFrame<K, V> {
struct MapChainFrame {
info: BlockInfo,
map: HashMap<K, V>,
map: HashMap<Name, SyntaxExtension>,
}
#[unsafe_destructor]
impl Drop for MapChainFrame {
fn drop(&mut self) {
// make sure that syntax extension dtors run before we drop the libs
self.map.clear();
}
}
// Only generic to make it easy to test
pub struct MapChain<K, V> {
priv chain: ~[MapChainFrame<K, V>],
pub struct SyntaxEnv {
priv chain: ~[MapChainFrame],
}
impl<K: Hash+Eq, V> MapChain<K, V> {
pub fn new() -> MapChain<K, V> {
let mut map = MapChain { chain: ~[] };
impl SyntaxEnv {
pub fn new() -> SyntaxEnv {
let mut map = SyntaxEnv { chain: ~[] };
map.push_frame();
map
}
@ -486,7 +507,7 @@ impl<K: Hash+Eq, V> MapChain<K, V> {
self.chain.pop();
}
fn find_escape_frame<'a>(&'a mut self) -> &'a mut MapChainFrame<K, V> {
fn find_escape_frame<'a>(&'a mut self) -> &'a mut MapChainFrame {
for (i, frame) in self.chain.mut_iter().enumerate().invert() {
if !frame.info.macros_escape || i == 0 {
return frame
@ -495,7 +516,7 @@ impl<K: Hash+Eq, V> MapChain<K, V> {
unreachable!()
}
pub fn find<'a>(&'a self, k: &K) -> Option<&'a V> {
pub fn find<'a>(&'a self, k: &Name) -> Option<&'a SyntaxExtension> {
for frame in self.chain.iter().invert() {
match frame.map.find(k) {
Some(v) => return Some(v),
@ -505,49 +526,15 @@ impl<K: Hash+Eq, V> MapChain<K, V> {
None
}
pub fn insert(&mut self, k: K, v: V) {
pub fn insert(&mut self, k: Name, v: SyntaxExtension) {
self.find_escape_frame().map.insert(k, v);
}
pub fn insert_macro_crate(&mut self, lib: DynamicLibrary) {
self.find_escape_frame().info.macro_crates.push(lib);
}
pub fn info<'a>(&'a mut self) -> &'a mut BlockInfo {
&mut self.chain[self.chain.len()-1].info
}
}
#[cfg(test)]
mod test {
use super::MapChain;
#[test]
fn testenv() {
let mut m = MapChain::new();
let (a,b,c,d) = ("a", "b", "c", "d");
m.insert(1, a);
assert_eq!(Some(&a), m.find(&1));
m.push_frame();
m.info().macros_escape = true;
m.insert(2, b);
assert_eq!(Some(&a), m.find(&1));
assert_eq!(Some(&b), m.find(&2));
m.pop_frame();
assert_eq!(Some(&a), m.find(&1));
assert_eq!(Some(&b), m.find(&2));
m.push_frame();
m.push_frame();
m.info().macros_escape = true;
m.insert(3, c);
assert_eq!(Some(&c), m.find(&3));
m.pop_frame();
assert_eq!(Some(&c), m.find(&3));
m.pop_frame();
assert_eq!(None, m.find(&3));
m.push_frame();
m.insert(4, d);
m.pop_frame();
assert_eq!(None, m.find(&4));
}
}

4
src/libsyntax/ext/build.rs
View file

@ -236,7 +236,7 @@ pub trait AstBuilder {
vis: ast::Visibility, path: ~[ast::Ident]) -> ast::ViewItem;
}
impl AstBuilder for ExtCtxt {
impl<'a> AstBuilder for ExtCtxt<'a> {
fn path(&self, span: Span, strs: ~[ast::Ident]) -> ast::Path {
self.path_all(span, false, strs, opt_vec::Empty, ~[])
}
@ -896,7 +896,7 @@ impl AstBuilder for ExtCtxt {
}
struct Duplicator<'a> {
cx: &'a ExtCtxt,
cx: &'a ExtCtxt<'a>,
}
impl<'a> Folder for Duplicator<'a> {

2
src/libsyntax/ext/deriving/generic.rs
View file

@ -190,7 +190,7 @@ mod ty;
pub struct TraitDef<'a> {
/// The extension context
cx: &'a ExtCtxt,
cx: &'a ExtCtxt<'a>,
/// The span for the current #[deriving(Foo)] header.
span: Span,

110
src/libsyntax/ext/expand.rs
View file

@ -30,6 +30,7 @@ use visit::Visitor;
use util::small_vector::SmallVector;
use std::vec;
use std::unstable::dynamic_lib::DynamicLibrary;
pub fn expand_expr(e: @ast::Expr, fld: &mut MacroExpander) -> @ast::Expr {
match e.node {
@ -365,13 +366,79 @@ pub fn expand_item_mac(it: @ast::Item, fld: &mut MacroExpander)
// yikes... no idea how to apply the mark to this. I'm afraid
// we're going to have to wait-and-see on this one.
fld.extsbox.insert(intern(name), ext);
SmallVector::zero()
if attr::contains_name(it.attrs, "macro_export") {
SmallVector::one(it)
} else {
SmallVector::zero()
}
}
};
fld.cx.bt_pop();
return items;
}
// load macros from syntax-phase crates
pub fn expand_view_item(vi: &ast::ViewItem,
fld: &mut MacroExpander)
-> ast::ViewItem {
let should_load = vi.attrs.iter().any(|attr| {
"phase" == attr.name() &&
attr.meta_item_list().map_or(false, |phases| {
attr::contains_name(phases, "syntax")
})
});
if should_load {
load_extern_macros(vi, fld);
}
noop_fold_view_item(vi, fld)
}
fn load_extern_macros(crate: &ast::ViewItem, fld: &mut MacroExpander) {
let MacroCrate { lib, cnum } = fld.cx.loader.load_crate(crate);
let exported_macros = fld.cx.loader.get_exported_macros(cnum);
for &it in exported_macros.iter() {
expand_item_mac(it, fld);
}
let path = match lib {
Some(path) => path,
None => return
};
// Make sure the path contains a / or the linker will search for it.
// If path is already absolute this is a no-op.
let path = Path::new(".").join(path);
let registrar = match fld.cx.loader.get_registrar_symbol(cnum) {
Some(registrar) => registrar,
None => return
};
let lib = match DynamicLibrary::open(Some(&path)) {
Ok(lib) => lib,
Err(err) => fld.cx.span_fatal(crate.span, err)
};
unsafe {
let registrar: MacroCrateRegistrationFun = match lib.symbol(registrar) {
Ok(registrar) => registrar,
Err(err) => fld.cx.span_fatal(crate.span, err)
};
registrar(|name, extension| {
let extension = match extension {
NormalTT(ext, _) => NormalTT(ext, Some(crate.span)),
IdentTT(ext, _) => IdentTT(ext, Some(crate.span)),
ItemDecorator(ext) => ItemDecorator(ext),
};
fld.extsbox.insert(name, extension);
});
}
fld.extsbox.insert_macro_crate(lib);
}
// expand a stmt
pub fn expand_stmt(s: &Stmt, fld: &mut MacroExpander) -> SmallVector<@Stmt> {
// why the copying here and not in expand_expr?
@ -878,7 +945,7 @@ pub fn inject_std_macros(parse_sess: @parse::ParseSess,
pub struct MacroExpander<'a> {
extsbox: SyntaxEnv,
cx: &'a mut ExtCtxt,
cx: &'a mut ExtCtxt<'a>,
}
impl<'a> Folder for MacroExpander<'a> {
@ -894,6 +961,10 @@ impl<'a> Folder for MacroExpander<'a> {
expand_item(item, self)
}
fn fold_view_item(&mut self, vi: &ast::ViewItem) -> ast::ViewItem {
expand_view_item(vi, self)
}
fn fold_stmt(&mut self, stmt: &ast::Stmt) -> SmallVector<@ast::Stmt> {
expand_stmt(stmt, self)
}
@ -908,9 +979,10 @@ impl<'a> Folder for MacroExpander<'a> {
}
pub fn expand_crate(parse_sess: @parse::ParseSess,
loader: &mut CrateLoader,
cfg: ast::CrateConfig,
c: Crate) -> Crate {
let mut cx = ExtCtxt::new(parse_sess, cfg.clone());
let mut cx = ExtCtxt::new(parse_sess, cfg.clone(), loader);
let mut expander = MacroExpander {
extsbox: syntax_expander_table(),
cx: &mut cx,
@ -1076,6 +1148,7 @@ mod test {
use codemap::Spanned;
use fold;
use fold::*;
use ext::base::{CrateLoader, MacroCrate};
use parse;
use parse::token::{fresh_mark, gensym, intern, ident_to_str};
use parse::token;
@ -1119,6 +1192,22 @@ mod test {
}
}
struct ErrLoader;
impl CrateLoader for ErrLoader {
fn load_crate(&mut self, _: &ast::ViewItem) -> MacroCrate {
fail!("lolwut")
}
fn get_exported_macros(&mut self, _: ast::CrateNum) -> ~[@ast::Item] {
fail!("lolwut")
}
fn get_registrar_symbol(&mut self, _: ast::CrateNum) -> Option<~str> {
fail!("lolwut")
}
}
// make sure that fail! is present
#[test] fn fail_exists_test () {
let src = @"fn main() { fail!(\"something appropriately gloomy\");}";
@ -1129,7 +1218,8 @@ mod test {
~[],sess);
let crate_ast = inject_std_macros(sess, ~[], crate_ast);
// don't bother with striping, doesn't affect fail!.
expand_crate(sess,~[],crate_ast);
let mut loader = ErrLoader;
expand_crate(sess,&mut loader,~[],crate_ast);
}
// these following tests are quite fragile, in that they don't test what
@ -1146,7 +1236,8 @@ mod test {
src,
~[],sess);
// should fail:
expand_crate(sess,~[],crate_ast);
let mut loader = ErrLoader;
expand_crate(sess,&mut loader,~[],crate_ast);
}
// make sure that macros can leave scope for modules
@ -1160,7 +1251,8 @@ mod test {
src,
~[],sess);
// should fail:
expand_crate(sess,~[],crate_ast);
let mut loader = ErrLoader;
expand_crate(sess,&mut loader,~[],crate_ast);
}
// macro_escape modules shouldn't cause macros to leave scope
@ -1173,7 +1265,8 @@ mod test {
src,
~[], sess);
// should fail:
expand_crate(sess,~[],crate_ast);
let mut loader = ErrLoader;
expand_crate(sess,&mut loader,~[],crate_ast);
}
#[test] fn std_macros_must_parse () {
@ -1281,7 +1374,8 @@ mod test {
fn expand_crate_str(crate_str: @str) -> ast::Crate {
let (crate_ast,ps) = string_to_crate_and_sess(crate_str);
// the cfg argument actually does matter, here...
expand_crate(ps,~[],crate_ast)
let mut loader = ErrLoader;
expand_crate(ps,&mut loader,~[],crate_ast)
}
//fn expand_and_resolve(crate_str: @str) -> ast::crate {

2
src/libsyntax/ext/format.rs
View file

@ -34,7 +34,7 @@ enum Position {
}
struct Context<'a> {
ecx: &'a mut ExtCtxt,
ecx: &'a mut ExtCtxt<'a>,
fmtsp: Span,
// Parsed argument expressions and the types that we've found so far for

2
src/libsyntax/ext/quote.rs
View file

@ -243,7 +243,7 @@ pub mod rt {
fn parse_tts(&self, s: @str) -> ~[ast::TokenTree];
}
impl ExtParseUtils for ExtCtxt {
impl<'a> ExtParseUtils for ExtCtxt<'a> {
fn parse_item(&self, s: @str) -> @ast::Item {
let res = parse::parse_item_from_source_str(

60
src/libsyntax/ext/registrar.rs Normal file
View file

@ -0,0 +1,60 @@
// Copyright 2012-2013 The Rust Project Developers. See the COPYRIGHT
// file at the top-level directory of this distribution and at
// http://rust-lang.org/COPYRIGHT.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.
use ast;
use attr;
use codemap::Span;
use diagnostic;
use visit;
use visit::Visitor;
struct MacroRegistrarContext {
registrars: ~[(ast::NodeId, Span)],
}
impl Visitor<()> for MacroRegistrarContext {
fn visit_item(&mut self, item: &ast::Item, _: ()) {
match item.node {
ast::ItemFn(..) => {
if attr::contains_name(item.attrs, "macro_registrar") {
self.registrars.push((item.id, item.span));
}
}
_ => {}
}
visit::walk_item(self, item, ());
}
}
pub fn find_macro_registrar(diagnostic: @diagnostic::SpanHandler,
crate: &ast::Crate) -> Option<ast::DefId> {
let mut ctx = MacroRegistrarContext { registrars: ~[] };
visit::walk_crate(&mut ctx, crate, ());
match ctx.registrars.len() {
0 => None,
1 => {
let (node_id, _) = ctx.registrars.pop();
Some(ast::DefId {
crate: ast::LOCAL_CRATE,
node: node_id
})
},
_ => {
diagnostic.handler().err("Multiple macro registration functions found");
for &(_, span) in ctx.registrars.iter() {
diagnostic.span_note(span, "one is here");
}
diagnostic.handler().abort_if_errors();
unreachable!();
}
}
}

37
src/libsyntax/fold.rs
View file

@ -63,20 +63,7 @@ pub trait Folder {
}
fn fold_view_item(&mut self, vi: &ViewItem) -> ViewItem {
let inner_view_item = match vi.node {
ViewItemExternMod(ref ident, string, node_id) => {
ViewItemExternMod(ident.clone(), string, self.new_id(node_id))
}
ViewItemUse(ref view_paths) => {
ViewItemUse(self.fold_view_paths(*view_paths))
}
};
ViewItem {
node: inner_view_item,
attrs: vi.attrs.map(|a| fold_attribute_(*a, self)),
vis: vi.vis,
span: self.new_span(vi.span),
}
noop_fold_view_item(vi, self)
}
fn fold_foreign_item(&mut self, ni: @ForeignItem) -> @ForeignItem {
@ -509,6 +496,28 @@ fn fold_variant_arg_<T: Folder>(va: &VariantArg, folder: &mut T) -> VariantArg {
}
}
pub fn noop_fold_view_item<T: Folder>(vi: &ViewItem, folder: &mut T)
-> ViewItem{
let inner_view_item = match vi.node {
ViewItemExternMod(ref ident,
string,
node_id) => {
ViewItemExternMod(ident.clone(),
string,
folder.new_id(node_id))
}
ViewItemUse(ref view_paths) => {
ViewItemUse(folder.fold_view_paths(*view_paths))
}
};
ViewItem {
node: inner_view_item,
attrs: vi.attrs.map(|a| fold_attribute_(*a, folder)),
vis: vi.vis,
span: folder.new_span(vi.span),
}
}
pub fn noop_fold_block<T: Folder>(b: P<Block>, folder: &mut T) -> P<Block> {
let view_items = b.view_items.map(|x| folder.fold_view_item(x));
let stmts = b.stmts.iter().flat_map(|s| folder.fold_stmt(*s).move_iter()).collect();

1
src/libsyntax/lib.rs
View file

@ -68,6 +68,7 @@ pub mod ext {
pub mod asm;
pub mod base;
pub mod expand;
pub mod registrar;
pub mod quote;

16
src/test/auxiliary/macro_crate_def_only.rs Normal file
View file

@ -0,0 +1,16 @@
// Copyright 2014 The Rust Project Developers. See the COPYRIGHT
// file at the top-level directory of this distribution and at
// http://rust-lang.org/COPYRIGHT.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.
#[feature(macro_rules)];
#[macro_export]
macro_rules! make_a_5(
() => (5)
)

42
src/test/auxiliary/macro_crate_test.rs Normal file
View file

@ -0,0 +1,42 @@
// Copyright 2013-2014 The Rust Project Developers. See the COPYRIGHT
// file at the top-level directory of this distribution and at
// http://rust-lang.org/COPYRIGHT.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.
#[feature(globs, macro_registrar, macro_rules)];
extern mod syntax;
use syntax::ast::{Name, TokenTree};
use syntax::codemap::Span;
use syntax::ext::base::*;
use syntax::parse::token;
#[macro_export]
macro_rules! exported_macro (() => (2))
macro_rules! unexported_macro (() => (3))
#[macro_registrar]
pub fn macro_registrar(register: |Name, SyntaxExtension|) {
register(token::intern("make_a_1"),
NormalTT(~SyntaxExpanderTT {
expander: SyntaxExpanderTTExpanderWithoutContext(expand_make_a_1),
span: None,
},
None));
}
pub fn expand_make_a_1(cx: &mut ExtCtxt, sp: Span, tts: &[TokenTree]) -> MacResult {
if !tts.is_empty() {
cx.span_fatal(sp, "make_a_1 takes no arguments");
}
MRExpr(quote_expr!(cx, 1i))
}
pub fn foo() {}

15
src/test/compile-fail/gated-macro_registrar.rs Normal file
View file

@ -0,0 +1,15 @@
// Copyright 2013 The Rust Project Developers. See the COPYRIGHT
// file at the top-level directory of this distribution and at
// http://rust-lang.org/COPYRIGHT.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.
// the registration function isn't typechecked yet
#[macro_registrar]
pub fn registrar() {} //~ ERROR cross-crate macro exports are experimental
fn main() {}

17
src/test/compile-fail/gated-phase.rs Normal file
View file

@ -0,0 +1,17 @@
// Copyright 2013 The Rust Project Developers. See the COPYRIGHT
// file at the top-level directory of this distribution and at
// http://rust-lang.org/COPYRIGHT.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.
// aux-build:macro_crate_test.rs
#[phase(syntax)]
//~^ ERROR compile time crate loading is experimental and possibly buggy
extern mod macro_crate_test;
fn main() {}

21
src/test/compile-fail/macro-crate-unexported-macro.rs Normal file
View file

@ -0,0 +1,21 @@
// Copyright 2014 The Rust Project Developers. See the COPYRIGHT
// file at the top-level directory of this distribution and at
// http://rust-lang.org/COPYRIGHT.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.
// aux-build:macro_crate_test.rs
// xfail-stage1
#[feature(phase)];
#[phase(syntax)]
extern mod macro_crate_test;
fn main() {
assert_eq!(3, unexported_macro!()); //~ ERROR macro undefined: 'unexported_macro'
}

16
src/test/compile-fail/macro-crate-unknown-crate.rs Normal file
View file

@ -0,0 +1,16 @@
// Copyright 2013 The Rust Project Developers. See the COPYRIGHT
// file at the top-level directory of this distribution and at
// http://rust-lang.org/COPYRIGHT.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.
#[feature(phase)];
#[phase(syntax)]
extern mod doesnt_exist; //~ ERROR can't find crate
fn main() {}

22
src/test/compile-fail/multiple-macro-registrars.rs Normal file
View file

@ -0,0 +1,22 @@
// Copyright 2013 The Rust Project Developers. See the COPYRIGHT
// file at the top-level directory of this distribution and at
// http://rust-lang.org/COPYRIGHT.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.
// error-pattern: Multiple macro registration functions found
#[feature(macro_registrar)];
// the registration function isn't typechecked yet
#[macro_registrar]
pub fn one() {}
#[macro_registrar]
pub fn two() {}
fn main() {}

24
src/test/compile-fail/phase-syntax-doesnt-resolve.rs Normal file
View file

@ -0,0 +1,24 @@
// Copyright 2013 The Rust Project Developers. See the COPYRIGHT
// file at the top-level directory of this distribution and at
// http://rust-lang.org/COPYRIGHT.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.
// aux-build:macro_crate_test.rs
// xfail-stage1
#[feature(phase)];
#[phase(syntax)]
extern mod macro_crate_test;
fn main() {
macro_crate_test::foo();
//~^ ERROR unresolved name
//~^^ ERROR use of undeclared module `macro_crate_test`
//~^^^ ERROR unresolved name `macro_crate_test::foo`.
}

21
src/test/run-pass/macro-crate-def-only.rs Normal file
View file

@ -0,0 +1,21 @@
// Copyright 2014 The Rust Project Developers. See the COPYRIGHT
// file at the top-level directory of this distribution and at
// http://rust-lang.org/COPYRIGHT.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.
// aux-build:macro_crate_def_only.rs
// xfail-fast
#[feature(phase)];
#[phase(syntax)]
extern mod macro_crate_def_only;
pub fn main() {
assert_eq!(5, make_a_5!());
}

23
src/test/run-pass/macro-crate.rs Normal file
View file

@ -0,0 +1,23 @@
// Copyright 2013-2014 The Rust Project Developers. See the COPYRIGHT
// file at the top-level directory of this distribution and at
// http://rust-lang.org/COPYRIGHT.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.
// aux-build:macro_crate_test.rs
// xfail-stage1
// xfail-fast
#[feature(phase)];
#[phase(syntax)]
extern mod macro_crate_test;
pub fn main() {
assert_eq!(1, make_a_1!());
assert_eq!(2, exported_macro!());
}

23
src/test/run-pass/phase-syntax-link-does-resolve.rs Normal file
View file

@ -0,0 +1,23 @@
// Copyright 2013 The Rust Project Developers. See the COPYRIGHT
// file at the top-level directory of this distribution and at
// http://rust-lang.org/COPYRIGHT.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.
// aux-build:macro_crate_test.rs
// xfail-stage1
// xfail-fast
#[feature(phase)];
#[phase(syntax, link)]
extern mod macro_crate_test;
fn main() {
assert_eq!(1, make_a_1!());
macro_crate_test::foo();
}