Some metrics improvements and timeline reporting #4451
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Robert (Bobby) Evans <bobby@apache.org>
revans2
added
task
gerashegalov
previously approved these changes
Jan 3, 2022
| yStart: Long, | ||
| minStart: Long, | ||
| fileWriter: ToolTextFileWriter): Unit = { | ||
| val x = xStart + (startTime - minStart)/MS_PER_PIXEL |
docs/tuning-guide.md
Outdated
| | Key | Name | Description | | ||
| |------------------|--------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | ||
| | bufferTime | buffer time | Time spent buffering input from file data sources. This buffering time happens on the CPU, typically with no GPU semaphore held. | | ||
| | readFsTime | time to read fs data | Time spent actually reading the data and writing it to on heap memory. This is a part of `bufferTime` | |
There was a problem hiding this comment.
hyphenated spelling on-heap , off-heap is easier to parse
|
build |
|
Converting to draft because I found some issues with op time for join that I want to understand better. |
|
build |
jlowe
reviewed
Jan 4, 2022
sql-plugin/src/main/scala/com/nvidia/spark/rapids/AbstractGpuJoinIterator.scala
Show resolved
Hide resolved
tools/src/main/scala/com/nvidia/spark/rapids/tool/profiling/GenerateTimeline.scala
Show resolved
Hide resolved
sql-plugin/src/main/scala/com/nvidia/spark/rapids/ColumnarOutputWriter.scala
Show resolved
Hide resolved
jlowe
approved these changes
Jan 4, 2022
|
build |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a result of trying to find a heuristic to optimize parquet/orc splits and also looking at buffering times to try and understand if there are more optimizations we could do for HDFS/other distributed file systems.
It fixes some metrics and offers a way to visualize the metrics in the timeline view from the profiling tool. The colors on the timeline were already really bad, and this does not help at all.
I don't consider this completely done because I have not documented the metrics reporting yet. I have not done this because I wasn't sure if the colors I have picked are okay. Also I was not sure if we wanted to put in a pattern in addition to a color to make it simpler to see. I also am not sure if this is something we want to have on by default, especially because the semaphore time only happens when debug metrics are enabled. Here is a high level overview.
The bottom half of each task shows the amount of time taken as reported by various metrics.
feedback is appreciated.