Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Give remote-test-client a configurable timeout #126959

Conversation

Hoverbear
Copy link
Contributor

I'm doing some work with a remote host running tests using x.py's remote test runner system.

After building the remote-test-server with:

./x build src/tools/remote-test-server --target aarch64-unknown-linux-gnu

I can transfer the remote-test-server to the remote and set up a port forwarded SSH connection, then I run:

TEST_DEVICE_ADDR=127.0.0.1:12345 ./x.py test library/core --target aarch64-unknown-linux-gnu

This works great if the host is nearby, however if the average round trip time is over 100ms (the timeout), it creates problems as it silently trips the timeout.

This PR checks an environment variable for a user configured timeout, defaulting to the old setting of 100 if it's not present. I picked the environment name because it seemed obvious, but there may be another option more suitable!

Also included is an explicit panic with a (hopefully) useful message if the deadline does get tripped. If it's more appropriate, I could make this a non-panic and simply a printed warning.

Testing

To test, you could run the remote-test-server similar to as described in https://rustc-dev-guide.rust-lang.org/tests/running.html#running-tests-on-a-remote-machine.

./x build src/tools/remote-test-server --stage 2
./build/x86_64-unknown-linux-gnu/stage2-tools/x86_64-unknown-linux-gnu/release/remote-test-server -v --bind 127.0.0.1:12345

Then, in another terminal:

TCP_TIMEOUT=300 TEST_DEVICE_ADDR=127.0.0.1:12345 ./x.py test tests/ui

To trigger the timeout, you'll need to introduce latency into your loopback and tune the timeout down to something below that.

That message looks like this:

    Finished `release` profile [optimized] target(s) in 0.14s
Connecting to remote device 127.0.0.1:12345 ...
thread 'main' panicked at src/tools/remote-test-client/src/main.rs:91:72:
TCP timeout of 100ms exceeded, set `TCP_TIMEOUT` to a larger value
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Build completed unsuccessfully in 0:00:07

@rustbot
Copy link
Collaborator

rustbot commented Jun 25, 2024

r? @Mark-Simulacrum

rustbot has assigned @Mark-Simulacrum.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@rustbot rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 25, 2024
match client.read_exact(&mut b) {
Ok(_) => break,
Err(e) if e.kind() == io::ErrorKind::WouldBlock => panic!(
"TCP timeout of {timeout}ms exceeded, set `{TCP_TIMEOUT_ENV}` to a larger value"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would be a more specific and appropriate error, yes! I wonder why I saw what I did while testing...

@Mark-Simulacrum
Copy link
Member

I think I'd expect us to just increase the timeout rather than making it configurable. I'm not sure there's any significant value of 100ms vs. something like 2 seconds (which seems likely to be enough for your use case?)

@Hoverbear
Copy link
Contributor Author

I'm perfectly happy to simply increase the timeout to 2s! Let me do that instead!

@Hoverbear
Copy link
Contributor Author

Closing in favor of #127246

@Hoverbear Hoverbear closed this Jul 2, 2024
jhpratt added a commit to jhpratt/rust that referenced this pull request Jul 3, 2024
…ient-has-longer-timeout, r=Mark-Simulacrum

Give remote-test-client a longer timeout

In rust-lang#126959, ``@Mark-Simulacrum`` suggested we simply extend the timeout of the `remote-test-client` instead of making it configurable. This is acceptable for my use case.

I'm doing some work with a remote host running tests using `x.py`'s remote test runner system.

After building the `remote-test-server` with:

```bash
./x build src/tools/remote-test-server --target aarch64-unknown-linux-gnu
```

I can transfer the `remote-test-server` to the remote and set up a port forwarded SSH connection, then I run:

```bash
TEST_DEVICE_ADDR=127.0.0.1:12345 ./x.py test library/core --target aarch64-unknown-linux-gnu
```

This works great if the host is nearby, however if the average round trip time is over 100ms (the timeout), it creates problems as it silently trips the timeout.

This PR extends that timeout to a less strict 2s.

r? ``@Mark-Simulacrum``
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Jul 3, 2024
Rollup merge of rust-lang#127246 - ferrocene:hoverbear/remote-test-client-has-longer-timeout, r=Mark-Simulacrum

Give remote-test-client a longer timeout

In rust-lang#126959, ``@Mark-Simulacrum`` suggested we simply extend the timeout of the `remote-test-client` instead of making it configurable. This is acceptable for my use case.

I'm doing some work with a remote host running tests using `x.py`'s remote test runner system.

After building the `remote-test-server` with:

```bash
./x build src/tools/remote-test-server --target aarch64-unknown-linux-gnu
```

I can transfer the `remote-test-server` to the remote and set up a port forwarded SSH connection, then I run:

```bash
TEST_DEVICE_ADDR=127.0.0.1:12345 ./x.py test library/core --target aarch64-unknown-linux-gnu
```

This works great if the host is nearby, however if the average round trip time is over 100ms (the timeout), it creates problems as it silently trips the timeout.

This PR extends that timeout to a less strict 2s.

r? ``@Mark-Simulacrum``
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants