Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Greatly speed up doctests by compiling compatible doctests in one file #123974

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
86833ee
Clean up rustdoc `make_test` function code
GuillaumeGomez Apr 10, 2024
587c522
Create DocTest type to handle generation of doctests
GuillaumeGomez Apr 11, 2024
4867760
Split doctests into categories
GuillaumeGomez Apr 12, 2024
78171c6
Compile all (compatible) doctests as one
GuillaumeGomez Apr 13, 2024
e022387
Remove `ignore_to_bool` function
GuillaumeGomez Apr 15, 2024
cce04bb
Set `ignore` value directly into `make_test`
GuillaumeGomez Apr 15, 2024
b8f0f2c
Add fallback in case the "all in one" doctests failed to compile
GuillaumeGomez Apr 15, 2024
56acb5d
Create `DocTestInfo` to improve API of `run_test`
GuillaumeGomez Apr 15, 2024
882d119
Correctly handle test args and move `ignore`d doctests into the one file
GuillaumeGomez Apr 15, 2024
45b075f
Fix case where AST failed to parse to give better errors
GuillaumeGomez Apr 15, 2024
cdb59a1
Correctly handle exit status
GuillaumeGomez Apr 15, 2024
e6f38c5
Bless rustdoc ui tests
GuillaumeGomez Apr 15, 2024
2c27d9c
Run doctests by batches instead of running them all at once when ther…
GuillaumeGomez Apr 16, 2024
76f70fa
Don't set `test_harness` to `true` by default as it would generate in…
GuillaumeGomez Apr 16, 2024
f235cbf
Ignore test using link which creates segfault
GuillaumeGomez Apr 16, 2024
b5ca844
Improve handling of `--persist-doctests` with combined doctests and u…
GuillaumeGomez Apr 17, 2024
e4b74f4
Run merged doctests as one instead of using batches
GuillaumeGomez Apr 29, 2024
7ea8909
Allow to merge doctests only starting the 2024 edition
GuillaumeGomez Apr 29, 2024
5a3d239
Add standalone doctest attribute
GuillaumeGomez Apr 30, 2024
847664a
Add documentation for the doctest `standalone` attribute
GuillaumeGomez Apr 30, 2024
24a23e1
Improve code
GuillaumeGomez May 3, 2024
88cbb78
Improve code and rustdoc book
GuillaumeGomez May 12, 2024
222f7dc
Add equivalent of rustdoc-ui test `failed-doctest-should-panic` for t…
GuillaumeGomez May 12, 2024
2e3c2aa
If there is any AST error with a doctest, we make it a standalone test
GuillaumeGomez May 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion library/std/src/keyword_docs.rs
Original file line number Diff line number Diff line change
Expand Up @@ -236,7 +236,7 @@ mod continue_keyword {}
/// fundamental compilation unit of Rust code, and can be seen as libraries or projects. More can
/// be read about crates in the [Reference].
///
/// ```rust ignore
/// ```rust,ignore (code sample)
/// extern crate rand;
/// extern crate my_crate as thing;
/// extern crate std; // implicitly added to the root of every Rust project
Expand Down
52 changes: 52 additions & 0 deletions src/doc/rustdoc/src/write-documentation/documentation-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -376,6 +376,58 @@ that the code sample should be compiled using the respective edition of Rust.
# fn foo() {}
```

Starting the 2024 edition[^edition-note], compatible doctests will be merged as one before being
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Starting the 2024 edition[^edition-note], compatible doctests will be merged as one before being
Starting in the 2024 edition[^edition-note], compatible doctests will be merged as one before being

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps also add a sentence explaining that it's for performance reasons? Just so people don't wonder why we're making the change. Could also be worth clarifying that if it fails, the individual tests will be rerun to give an exact error.

run. It means that they will share the process, so any change global/static variables will now
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
run. It means that they will share the process, so any change global/static variables will now
run. It means that they will share the process, so any change to global/static variables will now

impact the other doctests.

[^edition-note]: This is based on the edition of the whole crate, not the edition of the individual
test case that may be specified in its code attribute.

For example, if you have:

```rust
//! ```
//! foo::init();
//! ```

/// ```
/// foo::init();
/// ```
pub fn init() {
static mut IS_INIT: bool = false;

unsafe {
assert!(!IS_INIT);
IS_INIT = true;
}
}
```

If you run `rustdoc --test` on this code, it'll panic on the second doctest being
run because `IS_INIT` value is not `false` anymore.

This is where the `standalone` attribute comes in: it tells `rustdoc` that a doctest
should not be merged with the others and should be run in its own process. So the
previous code should use it:

```rust
//! ```standalone
//! foo::init();
//! ```

/// ```standalone
/// foo::init();
/// ```
pub fn init() {
static mut IS_INIT: bool = false;

unsafe {
assert!(!IS_INIT);
IS_INIT = true;
}
}
```

## Syntax reference

The *exact* syntax for code blocks, including the edge cases, can be found
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ The `link_arg_attribute` feature allows passing arguments into the linker
from inside of the source code. Order is preserved for link attributes as
they were defined on a single extern block:

```rust,no_run
```rust,ignore (linking to "c" segfaults)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was it already segfaulting, and the segfault just not getting caught by the test harness, or has something changed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually don't know how it could pass before. I opened #124043 about this.

#![feature(link_arg_attribute)]

#[link(kind = "link-arg", name = "--start-group")]
Expand Down
Loading
Loading