Stabilize macros in some more positions

[petrochenkov]

• Fn-like macros and attribute macros in extern blocks
• Fn-like procedural macros in type positions
• Attribute macros on inline modules (moved to Stabilize attribute macros on inline modules #64273)

Stabilization report: #63931 (comment).

Closes #49476
cc #54727

[petrochenkov]

Stabilization report

All the features below target Rust 1.39 (November 7th 2019).
Tests can be found in test files touched by this PR.

Fn-like macros and attribute macros in extern blocks

Macro attributes are now supported on items in extern blocks, the behavior is the same as for attributes on free/trait/impl items available on stable.

The feature name is #![feature(macros_in_extern)].
The feature was implemented in April 2018 in #49350 to fix issue #48747.
It didn't go through an RFC because the feature seemed to be a trivial extension consistent with already supported macros in item and item-like positions.

Fn-like procedural macros in type positions

Proc macros can now be used in type positions (type A = foo!();), the behavior is the same as for macro_rules macros in the same position, which are available on stable.
The difference is that proc macros on stable use call-site hygiene, while macro_rules use mixed call-site/def-site hygiene, but that difference only affects local variables, labels and $crate so it doesn't apply to types (or at least applies to types no more than to item-position macros, which are available on stable).

The feature is also a part of the polyphyletic #![feature(proc_macro_hygiene)].
The feature was implemented together with all other proc macros somewhere in 2017-2018.
It wasn't stabilized yet because it somehow got lost during the Macro 1.2 stabilization wave, and nobody actively pushed it after that.

[petrochenkov]

@Centril

it would be good to elaborate re. hygiene, how this may e.g. interact with expressions in type contexts, and give a recap (because issues re. hygiene isn't the best known subject...) of the issues wit proc macros expanding to expressions and why it isn't an issue here.

Proc macros in expressions and patterns have mostly philosophical issues - they can 1) consume local variables from their environment code and 2) can produce local variables consumable from their environment code.
Given that proc macros on stable can only use "unhygienic" Span::call_site hygiene, and unhygienic local variables are bad, proc macros in expression and patterns are prohibited in general.
Those philosophical issues do not apply to types, they can neither produce nor consume local variables.

The technical issue with expressions and patterns is that local variables stress the hygiene algorithm more and are more likely to hit some corner cases that may certainly exist because the hygiene algorithm does a few questionable things and is not fully understood.
That said, you still can work that feature gate around with stable item macros (https://crates.io/crates/proc-macro-hack), but some hope exists that proc-macro-hack is still a more limited exposure than arbitrary stable expression/pattern macros would be.

[petrochenkov]

As such, I would like us to consider these separately in 3 different stabilization PRs with associated reports & FCPs.

The features are pretty trivial (that's why they were selected for immediate stabilization in the first place), I can split the PR if the lang team wants, but that would mostly create extra work for everyone involved, IMO.

[Centril]

The features are pretty trivial (that's why they were selected for immediate stabilization in the first place), I can split the PR if the lang team wants, but that would mostly create extra work for everyone involved, IMO.

Fair enough; the types bit is the only bit I currently have worries about (see below).

Those philosophical issues do not apply to types, they can neither produce nor consume local variables.

How does this affect interactions with VLAs in the style of #48055 or run-time-value-dependent typing more generally? For example, let x: type_mac!() = ...; looks like it could "consume" a local let n if unhygienic and producing [u8; dyn n] or some such?

[Centril]

r? @Centril (r=me on the code itself once FCP completes)

[petrochenkov]

@Centril

How does this affect interactions with VLAs in the style of #48055 or run-time-value-dependent typing more generally? For example, let x: type_mac!() = ...; looks like it could "consume" a local let n if unhygienic and producing [u8; dyn n] or some such?

To clarify, Span::call_site hygiene generally works for local variables (I haven't seen bug reports for it at least), and already available on stable after jumping through a couple of hoops, e.g.

#[proc_macro]
fn item_macro(input) {
    "fn item() {
        ${input}
        let y = x; // OK    
    }"
}

item_macro!(let x = 10);

As I understand, the nrc's desire was to limit it (#52121 (comment)) rather than prohibit completely.
So further minor leaks like [u8; dyn n] should be ok.

[Centril]

First, sorry about the delay here but I think I need to understand better...

As I understand, the nrc's desire was to limit it (#52121 (comment)) rather than prohibit completely.

cc @nrc, @pnkfelix (maybe back from vacation soon?), @samth, and @rust-lang/lang

So further minor leaks like [u8; dyn n] should be ok.

So I feel a distinct lack of understanding re. what minor constitutes here. Would e.g. [u8; dyn { n = 3; n }] change anything?

[nrc]

As I understand, the nrc's desire was to limit it (#52121 (comment)) rather than prohibit completely.
So further minor leaks like [u8; dyn n] should be ok.

To clarify, I don't have a specific objection, just a general level of uncertainty about our macro hygiene in general. I'd prefer to not stabilise anything until we resolve that uncertainty, but that might have happened since I was last involved (or might be unrealistic to expect).

[petrochenkov]

So I feel a distinct lack of understanding re. what minor constitutes here. Would e.g. [u8; dyn { n = 3; n }] change anything?

Minor as in amount of code it's used in, so it less likely to hit corner cases statistically.
(n = 3 doesn't change anything.)
Anyway, we can feature gate dyn n in expansions specifically, if it's ever implemented.

[eddyb]

[T; dyn expr] in types isn't at all a thing that I've ever seen proposed.

The VLA(-like) proposal was for [expr; dyn expr] evaluating to a [T] value.

Runtime expressions in types amount to dependent typing, which we can all agree is far away, if ever, and for the time being these proc macros don't really have issues like that.

---

That said, the interesting case IMO is this:

macro_rules! foo {
    ($x:ident, $e:expr) => ([(); {
        let $x = 0;
        let x = 1;
        $e
    }])
}

macro_rules! bar {
    ($x:ident, $e:expr) => ([(); {
        let $x = 0;
        let $x = 1;
        $e
    }])
}

const _: foo!(x, x + 1) = [()];
const _: bar!(x, x + 1) = [(), ()];

If you were to implement either foo or bar as proc macros, would they both behave like bar?

However, even if that is the case, since the expression using x must be passed into the macro for the distinction to matter, the proc macro could very well "scavenge" for the right identifier in the input to produce the unhygienic behavior anyway.

So it does help that types are "closed expressions", wrt local bindings in scope.

[petrochenkov]

I've just noticed that this PR stabilizes the next code as well:

#[my_attr]
mod m {
    mod n; // Out-of-line module, how should `my_attr` see it in its input?
}

So, we additionally need to feature gate out-of-line modules in attr/derive input.

This may affect stable in corner cases:

#[my_attr]
fn m() {
    #[path = "zzz.rs"]
    mod n;
}

UPDATE: Done in #64273.

[rfcbot]

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

The RFC will be merged soon.

[petrochenkov]

@Centril
Rebased, the stabilization version is bumped to 1.40.

[petrochenkov]

@bors r=Centril
(This was actually r+'d in #63931 (comment).)

[bors]

📌 Commit 5ae38bb has been approved by Centril