Refactor to stop inheriting from HashJoin #1321
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
build |
andygrove
commented
Dec 8, 2020
| @@ -1,5 +1,5 @@ | |||
| /* | |||
| * Copyright (c) 2019-2020, NVIDIA CORPORATION. | |||
| * Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved. | |||
There was a problem hiding this comment.
Github is seeing this as a file move but I actually manually created this new file.
There was a problem hiding this comment.
git looks for diffs and tries to guess what was a move and what was not.
|
Overall it looks good and I love that we deleted 700 lines of uneeded code. |
Signed-off-by: Andy Grove <andygrove@nvidia.com>
andygrove
force-pushed
the
no-extend-hash-join
branch
from
43855e5
to
dd8c6b6
Compare
andygrove
changed the title
|
build |
revans2
previously approved these changes
Dec 9, 2020
| @@ -27,8 +27,9 @@ import org.apache.spark.sql.catalyst.optimizer.{BuildLeft, BuildRight, BuildSide | |||
| import org.apache.spark.sql.catalyst.plans.JoinType | |||
| import org.apache.spark.sql.catalyst.plans.physical.{Distribution, HashClusteredDistribution} | |||
| import org.apache.spark.sql.execution.{BinaryExecNode, SparkPlan} | |||
| import org.apache.spark.sql.execution.joins.ShuffledHashJoinExec | |||
| import org.apache.spark.sql.execution.joins.{ShuffledHashJoinExec} | |||
There was a problem hiding this comment.
nit don't need {} for a single import
There was a problem hiding this comment.
Thanks. There are also a couple unused imports after this refactor so I will push a commit to clean this up.
Signed-off-by: Andy Grove <andygrove@nvidia.com>
|
build |
revans2
approved these changes
Dec 9, 2020
nartal1
pushed a commit
to nartal1/spark-rapids
that referenced
this pull request
Jun 9, 2021
* Move GpuHashJoin out of shims and stop extending HashJoin Signed-off-by: Andy Grove <andygrove@nvidia.com> * clean up imports Signed-off-by: Andy Grove <andygrove@nvidia.com>
nartal1
pushed a commit
to nartal1/spark-rapids
that referenced
this pull request
Jun 9, 2021
* Move GpuHashJoin out of shims and stop extending HashJoin Signed-off-by: Andy Grove <andygrove@nvidia.com> * clean up imports Signed-off-by: Andy Grove <andygrove@nvidia.com>
tgravescs
pushed a commit
to tgravescs/spark-rapids
that referenced
this pull request
Nov 30, 2023
compute-saniziter is a tool can detect some GPU memory relevant issues, this PR is to enable it in memory check mode for the JNI unit tests. Some explanation for the parameters of the Compute Sanitizer. --log-file should be used to avoid a corrupting output issue from the surefire plugin. --error-exitcode is used to fail the build process if any error is caught by the Compute Sanitizer. --launch-timeout is set to 10 minutes, and it should be enough since we monitor only the forked test processes. --------- Signed-off-by: Liangcai Li <firestarmanllc@gmail.com> Co-authored-by: Jason Lowe <jlowe@nvidia.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Signed-off-by: Andy Grove andygrove@nvidia.com
Move
GpuHashJoinout of the shims and do not extend from Spark'sHashJoin.