batch small buffers when spilling via GDS #2295
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Rong Ou <rong.ou@gmail.com>
rongou
added
feature request
jlowe
requested changes
Apr 29, 2021
sql-plugin/src/main/scala/com/nvidia/spark/rapids/RapidsGdsStore.scala
Outdated
Show resolved
Hide resolved
sql-plugin/src/main/scala/com/nvidia/spark/rapids/RapidsGdsStore.scala
Outdated
Show resolved
Hide resolved
sql-plugin/src/main/scala/com/nvidia/spark/rapids/RapidsGdsStore.scala
Outdated
Show resolved
Hide resolved
sql-plugin/src/main/scala/com/nvidia/spark/rapids/RapidsGdsStore.scala
Outdated
Show resolved
Hide resolved
sql-plugin/src/main/scala/com/nvidia/spark/rapids/RapidsGdsStore.scala
Outdated
Show resolved
Hide resolved
sql-plugin/src/main/scala/com/nvidia/spark/rapids/RapidsGdsStore.scala
Outdated
Show resolved
Hide resolved
sql-plugin/src/main/scala/com/nvidia/spark/rapids/RapidsGdsStore.scala
Outdated
Show resolved
Hide resolved
Signed-off-by: Rong Ou <rong.ou@gmail.com>
jlowe
reviewed
Apr 29, 2021
sql-plugin/src/main/scala/com/nvidia/spark/rapids/RapidsGdsStore.scala
Outdated
Show resolved
Hide resolved
sql-plugin/src/main/scala/com/nvidia/spark/rapids/RapidsGdsStore.scala
Outdated
Show resolved
Hide resolved
sql-plugin/src/main/scala/com/nvidia/spark/rapids/RapidsGdsStore.scala
Outdated
Show resolved
Hide resolved
sql-plugin/src/main/scala/com/nvidia/spark/rapids/RapidsGdsStore.scala
Outdated
Show resolved
Hide resolved
Signed-off-by: Rong Ou <rong.ou@gmail.com>
jlowe
reviewed
Apr 30, 2021
There was a problem hiding this comment.
It would be good to have at least some testing of this new functionality, for example a test that verifies that buffers below the threshold are collected into a single file, large buffers are not, and that the batched file is removed only after the last buffer that was in the batch is freed.
sql-plugin/src/main/scala/com/nvidia/spark/rapids/RapidsGdsStore.scala
Outdated
Show resolved
Hide resolved
abellina
requested changes
Apr 30, 2021
sql-plugin/src/main/scala/com/nvidia/spark/rapids/RapidsGdsStore.scala
Outdated
Show resolved
Hide resolved
sql-plugin/src/main/scala/com/nvidia/spark/rapids/RapidsGdsStore.scala
Outdated
Show resolved
Hide resolved
Signed-off-by: Rong Ou <rong.ou@gmail.com>
Signed-off-by: Rong Ou <rong.ou@gmail.com>
jlowe
reviewed
May 7, 2021
sql-plugin/src/main/scala/com/nvidia/spark/rapids/RapidsGdsStore.scala
Outdated
Show resolved
Hide resolved
sql-plugin/src/main/scala/com/nvidia/spark/rapids/RapidsGdsStore.scala
Outdated
Show resolved
Hide resolved
Signed-off-by: Rong Ou <rong.ou@gmail.com>
Signed-off-by: Rong Ou <rong.ou@gmail.com>
Signed-off-by: Rong Ou <rong.ou@gmail.com>
|
Added tests. PTAL |
|
build |
jlowe
approved these changes
May 10, 2021
abellina
approved these changes
May 10, 2021
nartal1
pushed a commit
to nartal1/spark-rapids
that referenced
this pull request
Jun 9, 2021
* batch small buffers when spilling via GDS Signed-off-by: Rong Ou <rong.ou@gmail.com> * address review feedback Signed-off-by: Rong Ou <rong.ou@gmail.com> * more review comments Signed-off-by: Rong Ou <rong.ou@gmail.com> * refactoring Signed-off-by: Rong Ou <rong.ou@gmail.com> * renaming Signed-off-by: Rong Ou <rong.ou@gmail.com> * reduce batch write buffer size Signed-off-by: Rong Ou <rong.ou@gmail.com> * remove SingleShotSpiller class Signed-off-by: Rong Ou <rong.ou@gmail.com> * add stream sync before reading from cufile Signed-off-by: Rong Ou <rong.ou@gmail.com> * add gds store tests Signed-off-by: Rong Ou <rong.ou@gmail.com>
nartal1
pushed a commit
to nartal1/spark-rapids
that referenced
this pull request
Jun 9, 2021
* batch small buffers when spilling via GDS Signed-off-by: Rong Ou <rong.ou@gmail.com> * address review feedback Signed-off-by: Rong Ou <rong.ou@gmail.com> * more review comments Signed-off-by: Rong Ou <rong.ou@gmail.com> * refactoring Signed-off-by: Rong Ou <rong.ou@gmail.com> * renaming Signed-off-by: Rong Ou <rong.ou@gmail.com> * reduce batch write buffer size Signed-off-by: Rong Ou <rong.ou@gmail.com> * remove SingleShotSpiller class Signed-off-by: Rong Ou <rong.ou@gmail.com> * add stream sync before reading from cufile Signed-off-by: Rong Ou <rong.ou@gmail.com> * add gds store tests Signed-off-by: Rong Ou <rong.ou@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Signed-off-by: Rong Ou rong.ou@gmail.com
When spilling via GDS, if the user specifies a large number of shuffle partitions, each shuffle buffer can be fairly small and very inefficient to write to disk individually. This PR adds a batch write buffer that collects these smaller buffers and writes them out in a single
cuFileWritecall. Preliminary results show this greatly increases the write throughput and reduces the overall query time.