Extend compiletest to support subdirectories #31749

nikomatsakis · 2016-02-18T10:24:13Z

You can now group tests into directories like run-pass/borrowck or compile-fail/borrowck. By default, all .rs files within any directory are considered tests: to ignore some directory, create a placeholder file called compiletest-ignore-dir (I had to do this for several existing directories).

r? @alexcrichton
cc @brson

nikomatsakis · 2016-02-18T10:24:46Z

Hopefully we can use this to make it so that compile-fail and run-pass can actually be listed in GH again rather than surpassing some "don't show me this" threshold. :)

pnkfelix · 2016-02-18T12:00:11Z

Sweet!

We should move all of the internal data checking tests that use rustc_fail into their own dedicated sub directory after this lands!

nikomatsakis · 2016-02-18T14:27:08Z

@pnkfelix yeah, I was thinking that it'd be nice to:

move the existing ad-hoc subdirectories into their own subdirectories, so that things are more obviously grouped;
deprecate the existing auxiliary system for cross-crate tests and instead somehow package the crate together with subcrates in a subdirectory (that might require a bit more work though)

nikomatsakis · 2016-02-18T14:27:29Z

I want to do something like the latter point for incr. comp. tests, so I might wind up adding support for it, or at least the groundwork.

alexcrichton · 2016-02-18T17:56:08Z

@bors: r+ d3e480f

nikomatsakis · 2016-02-18T19:10:54Z

@bors r=alexcrichton

bors · 2016-02-18T19:10:55Z

📌 Commit d3e480f has been approved by alexcrichton

sanxiyn · 2016-02-19T01:38:31Z

Sorry to grumble here, but now I can't avoid feeling #20796 was unjustly closed.

pnkfelix · 2016-02-19T08:59:50Z

@sanxiyn fixed!

(I am kidding; I do understand your underlying message here. But, to be fair, these things happen, I don't think amy ill will was meant, and a closed issue with no comments beyond the description does not represent a huge amount of lost effort, at least compared to say if that had been a PR of yours that had been closed, right?)

bors · 2016-02-20T16:36:16Z

⌛ Testing commit d3e480f with merge 8caf5be...

bors · 2016-02-20T17:36:36Z

💔 Test failed - auto-win-gnu-64-nopt-t

steveklabnik · 2016-02-22T22:17:58Z

These errors seem very odd:

C:/bot/slave/auto-win-gnu-64-nopt-t/build/src/test/compile-fail/borrowck\borrowck-while-cond.rs:13:11: 13:12 error: use of possibly uninitialized variable: `x` [E0381]
C:/bot/slave/auto-win-gnu-64-nopt-t/build/src/test/compile-fail/borrowck\borrowck-while-cond.rs:13     while x { } //~ ERROR use of possibly uninitialized variable: `x`
                                                                                                             ^
C:/bot/slave/auto-win-gnu-64-nopt-t/build/src/test/compile-fail/borrowck\borrowck-while-cond.rs:13:11: 13:12 help: run `rustc --explain E0381` to see a detailed explanation
error: aborting due to previous error

It's right there that it's supposed to be checking for that exact error.

nikomatsakis · 2016-02-23T20:17:42Z

@steveklabnik it must be breaking because of the \ messing up a regex or something...

nikomatsakis · 2016-02-24T21:21:10Z

@bors r+

bors · 2016-02-24T21:21:11Z

📌 Commit c4b6037 has been approved by nikomatsakis

around a set of paths called `TestPaths` This commit is not quite standalone; it basically contains all the borrowing plumbing bits, the interesting stuff comes in the next commit.

subdirectory contains `compiletest-ignore-dir`, then ignore it.

as a test.

bors · 2016-02-25T00:10:35Z

⌛ Testing commit c4b6037 with merge 10b3a4e...

@alexcrichton

You can now group tests into directories like `run-pass/borrowck` or `compile-fail/borrowck`. By default, all `.rs` files within any directory are considered tests: to ignore some directory, create a placeholder file called `compiletest-ignore-dir` (I had to do this for several existing directories). r? @alexcrichton cc @brson

bors · 2016-02-25T01:19:48Z

💔 Test failed - auto-win-gnu-32-opt

nikomatsakis · 2016-02-25T08:31:39Z

@bors r=alexcrichton

bors · 2016-02-25T08:31:40Z

📌 Commit 588d5d1 has been approved by alexcrichton

bors · 2016-02-25T09:34:08Z

⌛ Testing commit 588d5d1 with merge df8c7a2...

Manishearth · 2016-02-25T09:38:27Z

(rollup)

bors · 2016-02-25T11:06:47Z

⌛ Testing commit 588d5d1 with merge a084ab3...

bors · 2016-02-25T11:09:04Z

⛄ The build was interrupted to prioritize another pull request.

bors · 2016-02-25T12:25:06Z

⚡ Previous build results for auto-mac-32-opt are reusable. Rebuilding only auto-linux-32-nopt-t, auto-linux-32-opt, auto-linux-64-debug-opt, auto-linux-64-nopt-t, auto-linux-64-opt, auto-linux-64-x-android-t, auto-linux-cross-opt, auto-linux-musl-64-opt, auto-mac-64-nopt-t, auto-mac-64-opt, auto-mac-ios-opt, auto-win-gnu-32-nopt-t, auto-win-gnu-32-opt, auto-win-gnu-64-nopt-t, auto-win-gnu-64-opt, auto-win-msvc-32-opt, auto-win-msvc-64-opt...

bors · 2016-02-25T13:43:57Z

💔 Test failed - auto-win-gnu-64-nopt-t

alexcrichton · 2016-02-25T17:07:08Z

@bors: retry

On Thu, Feb 25, 2016 at 5:44 AM, bors notifications@github.com wrote:

[image: 💔] Test failed - auto-win-gnu-64-nopt-t
http://buildbot.rust-lang.org/builders/auto-win-gnu-64-nopt-t/builds/3236

—
Reply to this email directly or view it on GitHub
#31749 (comment).

nikomatsakis · 2016-02-25T18:26:47Z

@alexcrichton I think I have a fix to this problem

alexcrichton · 2016-02-25T18:28:51Z

@bors: r-

Maybe I'll stop blindly retrying things one day

parallel execution

nikomatsakis · 2016-02-25T19:41:02Z

@bors r=alexcrichton

bors · 2016-02-25T19:41:03Z

📌 Commit a04b4b6 has been approved by alexcrichton

nikomatsakis · 2016-02-25T19:41:19Z

@bors r-

nikomatsakis · 2016-02-25T19:42:19Z

@bors r=alexcrichton

bors · 2016-02-25T19:42:20Z

📌 Commit d21434c has been approved by alexcrichton

@alexcrichton

You can now group tests into directories like `run-pass/borrowck` or `compile-fail/borrowck`. By default, all `.rs` files within any directory are considered tests: to ignore some directory, create a placeholder file called `compiletest-ignore-dir` (I had to do this for several existing directories). r? @alexcrichton cc @brson

bors · 2016-02-26T00:53:39Z

⌛ Testing commit d21434c with merge 15e9a95...

bors · 2016-02-26T02:54:56Z

☀️ Test successful - auto-linux-32-nopt-t, auto-linux-32-opt, auto-linux-64-debug-opt, auto-linux-64-nopt-t, auto-linux-64-opt, auto-linux-64-x-android-t, auto-linux-cross-opt, auto-linux-musl-64-opt, auto-mac-32-opt, auto-mac-64-nopt-t, auto-mac-64-opt, auto-mac-ios-opt, auto-win-gnu-32-nopt-t, auto-win-gnu-32-opt, auto-win-gnu-64-nopt-t, auto-win-gnu-64-opt, auto-win-msvc-32-opt, auto-win-msvc-64-opt

pitdicker · 2016-02-26T07:41:48Z

I think the problem with auto-win-gnu-64-nopt-t is the same I had with #29497.
In run_rustdoc_test there is a remove_dir_all where the result is ignored, directly followed by ensure_dir.
This is currently racy on Windows...

pitdicker · 2016-02-26T08:24:18Z

Oops, of course not (doctest gives it away). But I have a branch that should remove the race problem with create_dir_all. Time to finish it.

rust-highfive assigned alexcrichton Feb 18, 2016

nikomatsakis added 4 commits February 24, 2016 18:40

Generalize from passing around a single path to the test file to passing

7c2e5e3

around a set of paths called `TestPaths` This commit is not quite standalone; it basically contains all the borrowing plumbing bits, the interesting stuff comes in the next commit.

Recurse to find test files in any subdirectory of the base path. If a

c1ec32d

subdirectory contains `compiletest-ignore-dir`, then ignore it.

Add compiletest-ignore-dir to existing subdirectories.

1c850d8

Move the borrowck run-pass/compile-fail tests into their own directories

6660ad6

as a test.

translate backslashes to forward slashes in the expected strings

588d5d1

nikomatsakis force-pushed the compiletest-subdir branch from c4b6037 to 588d5d1 Compare February 25, 2016 08:31

Manishearth closed this Feb 25, 2016

Manishearth reopened this Feb 25, 2016

create the required directories while gathering tests rather than during

a04b4b6

parallel execution

add comment and stop creating directories in runtest

d21434c

bors merged commit d21434c into rust-lang:master Feb 26, 2016

bors mentioned this pull request Feb 26, 2016

Use box syntax in vec! macro #31797

Merged

nikomatsakis deleted the compiletest-subdir branch March 30, 2016 16:12

Extend compiletest to support subdirectories #31749

Extend compiletest to support subdirectories #31749

Conversation

nikomatsakis commented Feb 18, 2016

nikomatsakis commented Feb 18, 2016

pnkfelix commented Feb 18, 2016

nikomatsakis commented Feb 18, 2016

nikomatsakis commented Feb 18, 2016

alexcrichton commented Feb 18, 2016

nikomatsakis commented Feb 18, 2016

bors commented Feb 18, 2016

sanxiyn commented Feb 19, 2016

pnkfelix commented Feb 19, 2016

bors commented Feb 20, 2016

bors commented Feb 20, 2016

steveklabnik commented Feb 22, 2016

nikomatsakis commented Feb 23, 2016

nikomatsakis commented Feb 24, 2016

bors commented Feb 24, 2016

bors commented Feb 25, 2016

bors commented Feb 25, 2016

nikomatsakis commented Feb 25, 2016

bors commented Feb 25, 2016

bors commented Feb 25, 2016

Manishearth commented Feb 25, 2016

bors commented Feb 25, 2016

bors commented Feb 25, 2016

bors commented Feb 25, 2016

bors commented Feb 25, 2016

alexcrichton commented Feb 25, 2016

nikomatsakis commented Feb 25, 2016

alexcrichton commented Feb 25, 2016

nikomatsakis commented Feb 25, 2016

bors commented Feb 25, 2016

nikomatsakis commented Feb 25, 2016

nikomatsakis commented Feb 25, 2016

bors commented Feb 25, 2016

bors commented Feb 26, 2016

bors commented Feb 26, 2016

pitdicker commented Feb 26, 2016

pitdicker commented Feb 26, 2016