[SPARK-33607][SS][WEBUI] Input Rate timeline/histogram aren't rendered if built with Scala 2.13 #30546
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
This PR fixes an issue that the histogram and timeline aren't rendered in the
Streaming Query Statistics
page if we built Spark with Scala 2.13.The reason is
maxRecordRate
can beNaN
for Scala 2.13.The
NaN
is the result ofquery.recentProgress.map(_.inputRowsPerSecond).max
when the first element ofquery.recentProgress.map(_.inputRowsPerSecond)
isNaN
.Actually, the comparison logic for
Double
type was changed in Scala 2.13.scala/bug#12107
scala/scala#6410
So this issue happens as of Scala 2.13.
The root cause of the
NaN
is here.This
NaN
seems to be an initial value ofinputTimeSec
so I thinkDouble.PositiveInfinity
is suitable rather thanNaN
and this change can resolve this issue.Why are the changes needed?
To make sure we can use the histogram/timeline with Scala 2.13.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
First, I built with the following commands.
Then, ran the following query (this is brought from #30427 ).
Finally, I confirmed that the timeline and histogram are rendered.