Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEA] Have hashed shuffle match spark #938

Closed
revans2 opened this issue Oct 13, 2020 · 1 comment
Closed

[FEA] Have hashed shuffle match spark #938

revans2 opened this issue Oct 13, 2020 · 1 comment
Assignees
@revans2
Labels
feature request New feature or request P1 Nice to have for release

Comments

@revans2
Copy link
Collaborator

revans2 commented Oct 13, 2020

Is your feature request related to a problem? Please describe.
We have to jump through a number of hoops for joins because the hash used by cudf does not match bit for bit with the hash used by spark. If we get a murmur3 implementation that matches perfectly with spark #937 then we could implement shuffle partitioning in terms of it and match spark exactly.
@revans2 revans2 added feature request New feature or request ? - Needs Triage Need team to review and classify labels Oct 13, 2020
@revans2 revans2 mentioned this issue Oct 13, 2020
Closed
@sameerz sameerz added P1 Nice to have for release and removed ? - Needs Triage Need team to review and classify labels Oct 20, 2020
@revans2 revans2 self-assigned this Apr 29, 2021
@revans2
Copy link
Collaborator Author

revans2 commented Apr 29, 2021

I believe that this is done now.

@revans2 revans2 closed this as completed Apr 29, 2021
tgravescs pushed a commit to tgravescs/spark-rapids that referenced this issue Nov 30, 2023
@nvauto
Update submodule cudf to ac60656bc929c08f522d23328a515caa52cee989 (NV…
365a6a4 
…IDIA#938)

Signed-off-by: spark-rapids automation <70000568+nvauto@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@revans2 revans2

Labels
feature request New feature or request P1 Nice to have for release
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@revans2 @sameerz