Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEA] Add retry to GpuShuffleExchangeExec #8502

Closed
abellina opened this issue Jun 5, 2023 · 0 comments · Fixed by #9419
Closed

[FEA] Add retry to GpuShuffleExchangeExec #8502

abellina opened this issue Jun 5, 2023 · 0 comments · Fixed by #9419
Assignees
@winningsix
Labels
reliability Features to improve reliability or bugs that severly impact the reliability of the plugin

Comments

@abellina
Copy link
Collaborator

abellina commented Jun 5, 2023 •
edited by winningsix
Loading

Is your feature request related to a problem? Please describe.
The shuffle exchange exec calls into a partitioner in order to slice up incoming batches. We should be able to wrap the partitioner call in a withRetry at the very least, but perhaps we can do more (split) here and produce multiple batches per partition, at least the shuffle writers support that concept.

Tasks
Beta Give feedback
No tasks being tracked yet.
@abellina abellina added feature request New feature or request ? - Needs Triage Need team to review and classify reliability Features to improve reliability or bugs that severly impact the reliability of the plugin labels Jun 5, 2023
@abellina abellina mentioned this issue Jun 5, 2023
Open
@mattahrens mattahrens removed the ? - Needs Triage Need team to review and classify label Jun 7, 2023
@winningsix winningsix mentioned this issue Oct 12, 2023
Merged
@firestarman firestarman linked a pull request Oct 16, 2023 that will close this issue
Add retry to RoundRobin Partitioner and Range Partitioner #9419
Merged
@sameerz sameerz removed the feature request New feature or request label Oct 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@winningsix winningsix

Labels
reliability Features to improve reliability or bugs that severly impact the reliability of the plugin
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add retry to RoundRobin Partitioner and Range Partitioner winningsix/spark-rapids
4 participants
@abellina @winningsix @mattahrens @sameerz