Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] When tasks complete, cancel pending UCX requests #2478

Closed
abellina opened this issue May 21, 2021 · 0 comments · Fixed by #2553
Closed

[BUG] When tasks complete, cancel pending UCX requests #2478

abellina opened this issue May 21, 2021 · 0 comments · Fixed by #2553
Assignees
@abellina
Labels
bug Something isn't working P1 Nice to have for release performance A performance related task/issue shuffle things that impact the shuffle plugin
Milestone
May 24 - Jun 4

Comments

@abellina
Copy link
Collaborator

Splitting off a piece of pending requests when tasks complete (from #1178):

The lowest hanging fruit are client request that are waiting for bounce buffers: No resources have been acquired, and also no risk of a server having pending sends. So I think this part is doable today. This could cover a very large number of messages as well. Just thinking in terms of bounce buffer lengths.. we could have several bounce buffer lengths not ready to be received in the queue.

This issue is to track just this piece.
@abellina abellina added bug Something isn't working ? - Needs Triage Need team to review and classify performance A performance related task/issue labels May 21, 2021
@abellina abellina added this to the May 24 - Jun 4 milestone May 21, 2021
@abellina abellina added shuffle things that impact the shuffle plugin P1 Nice to have for release labels May 21, 2021
@abellina abellina self-assigned this May 21, 2021
@abellina abellina changed the title [BUG] when tasks that are driving a RapidsShuffleIterator complete, cancel its pending requests [BUG] When tasks complete, cancel pending UCX requests May 21, 2021
@sameerz sameerz removed the ? - Needs Triage Need team to review and classify label May 25, 2021
@abellina abellina mentioned this issue Jun 1, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@abellina abellina

Labels
bug Something isn't working P1 Nice to have for release performance A performance related task/issue shuffle things that impact the shuffle plugin
Projects
None yet
Milestone
May 24 - Jun 4
Development

Successfully merging a pull request may close this issue.

Cancel requests that are queued for a client/handler on error abellina/spark-rapids
2 participants
@abellina @sameerz