-
Notifications
You must be signed in to change notification settings - Fork 232
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Speed up test performance using pytest-xdist #1086
Conversation
Signed-off-by: Robert (Bobby) Evans <bobby@apache.org>
build |
build |
build |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so if I'm running the tests through maven and want to override the parallelism, I set TEST_PARALLEL=1 in the env?
You would want to add |
build |
Signed-off-by: Robert (Bobby) Evans <bobby@apache.org>
Signed-off-by: Robert (Bobby) Evans <bobby@apache.org>
Signed-off-by: Robert (Bobby) Evans <bobby@apache.org>
…IDIA#1086) Signed-off-by: spark-rapids automation <70000568+nvauto@users.noreply.github.com>
This covers the same work that was done in #1022 but finishes it.
It now will also autodetect the parallelism by default so we can ideally run as fast as possible in the normal case.
It also falls back to slower processing if pytest-xdist cannot be found.