Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

At BatchedTableCompressor.finish synchronize to allow for "right-size… #1248

Merged

Conversation

abellina
Copy link
Collaborator

@abellina abellina commented Dec 3, 2020

Addresses the P1 nature of #1245 for branch-0.3

…" copies to finish before it is potentially spilled

Signed-off-by: Alessandro Bellina <abellina@nvidia.com>
@abellina abellina requested a review from jlowe December 3, 2020 00:04
@jlowe jlowe added the shuffle things that impact the shuffle plugin label Dec 3, 2020
@jlowe jlowe added this to the Nov 23 - Dec 4 milestone Dec 3, 2020
@abellina abellina marked this pull request as ready for review December 3, 2020 00:05
@abellina
Copy link
Collaborator Author

abellina commented Dec 3, 2020

build

1 similar comment
@sameerz
Copy link
Collaborator

sameerz commented Dec 3, 2020

build

@abellina abellina merged commit ffd0d32 into NVIDIA:branch-0.3 Dec 3, 2020
@abellina abellina deleted the shuffle/synchronize_after_compression branch December 3, 2020 06:53
nartal1 pushed a commit to nartal1/spark-rapids that referenced this pull request Jun 9, 2021
…" copies to finish before it is potentially spilled (NVIDIA#1248)

Signed-off-by: Alessandro Bellina <abellina@nvidia.com>
nartal1 pushed a commit to nartal1/spark-rapids that referenced this pull request Jun 9, 2021
…" copies to finish before it is potentially spilled (NVIDIA#1248)

Signed-off-by: Alessandro Bellina <abellina@nvidia.com>
tgravescs pushed a commit to tgravescs/spark-rapids that referenced this pull request Nov 30, 2023
* Back port spark-specific murmur32 hash code from cudf.

* Run pre-commit to format files. We were behind a bit.

* Update pre-commit config to 16.0.1 to match cudf. Re-ran formatting.

* Change jni bindings to use the spark-rapids-jni implementation of murmur hash instead of the cudf version.  Brought over
cpp and java tests.

* Documentation fix.

* Fix cpp tests to actually call the spark_rapids_jni murmur hash.

* First pass at xxhash64. cpp tests passing.

* Improve cpp tests - null cases and more floating point edge cases.

* Add Java tests.

* Moved murmur32 hash implementaion from cudf to spark-rapids-jni

Signed-off-by: db <dbaranec@nvidia.com>

* PR review changes.

* Fix copyright data in Hash.java

* Enable 32 bit decimal hash test.

* Implement xxhash64 on the gpu

Signed-off-by: db <dbaranec@nvidia.com>

* Add missing newlines.

* PR review changes.

* Remove default xxhash64 class constructor.  Remove unused parameter (row index) from remaining constructor.

* Fix issues with merge.

* Rectify thirdparty/cudf issues.

* Revert inadvertent change to pom.xml

* PR review feedback changes.

---------

Signed-off-by: db <dbaranec@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
shuffle things that impact the shuffle plugin
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants