Update GpuDataSourceScanExec and GpuBroadcastExchangeExec to fix audit issues #1760
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…t issues Signed-off-by: Niranjan Artal <nartal@nvidia.com>
| @@ -265,7 +265,10 @@ abstract class GpuBroadcastExchangeExecBase( | |||
| @transient | |||
| private val timeout: Long = SQLConf.get.broadcastTimeout | |||
|
|
|||
| val _runId: UUID = UUID.randomUUID() | |||
| // Cancelling a SQL statement from Spark ThriftServer needs to cancel | |||
There was a problem hiding this comment.
yeah generally speaking would be better to separate issues in separate prs.
There was a problem hiding this comment.
note its fine for now but something to keep in mind for future
There was a problem hiding this comment.
Thanks @tgravescs and @gerashegalov. Will post separate PR's for separate issues in the future.
|
build |
tgravescs
reviewed
Feb 19, 2021
| @@ -138,6 +139,8 @@ trait SparkShims { | |||
| readFunction: (PartitionedFile) => Iterator[InternalRow], | |||
| filePartitions: Seq[FilePartition]): RDD[InternalRow] | |||
|
|
|||
| def getFileSourceMaxMetadataValueLength(sqlContext: SQLConf): Int | |||
There was a problem hiding this comment.
I would rename sqlContext to be sqlConf here because you aren't passing the entire sqlContext in
Signed-off-by: Niranjan Artal <nartal@nvidia.com>
|
build |
tgravescs
approved these changes
Feb 19, 2021
tgravescs
added a commit
to tgravescs/spark-rapids
that referenced
this pull request
Mar 22, 2021
…fix audit issues (NVIDIA#1760)" This reverts commit b4d996b. Signed-off-by: Thomas Graves <tgraves@nvidia.com>
nartal1
added a commit
to nartal1/spark-rapids
that referenced
this pull request
Jun 9, 2021
…t issues (NVIDIA#1760) Signed-off-by: Niranjan Artal <nartal@nvidia.com>
nartal1
added a commit
to nartal1/spark-rapids
that referenced
this pull request
Jun 9, 2021
…t issues (NVIDIA#1760) Signed-off-by: Niranjan Artal <nartal@nvidia.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This fixes #1460 and fixes #1731.
Update to GpuBroadcastExchangeExec.scala file fixes #1731.
Rest of the files are updated to fix #1460
Signed-off-by: Niranjan Artal nartal@nvidia.com