Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix auto merge conflict 5850 #5853

Merged
merged 3 commits into from
Jun 16, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Change log
Generated on 2022-06-14
Generated on 2022-06-16

## Release 22.06

Expand Down Expand Up @@ -143,6 +143,9 @@ Generated on 2022-06-14
### PRs
|||
|:---|:---|
|[#5848](https://github.com/NVIDIA/spark-rapids/pull/5848)|Update spark330shim to use released lib|
|[#5840](https://github.com/NVIDIA/spark-rapids/pull/5840)|[DOC] Updated RapidsConf to reflect the default value of `spark.rapids.sql.improvedFloatOps.enabled` [skip ci]|
|[#5816](https://github.com/NVIDIA/spark-rapids/pull/5816)|Update 22.06.0 changelog to latest [skip ci]|
|[#5795](https://github.com/NVIDIA/spark-rapids/pull/5795)|Update FAQ to include local jar deployment via extraClassPath [skip ci]|
|[#5802](https://github.com/NVIDIA/spark-rapids/pull/5802)|Update spark-rapids-jni.version to release 22.06.0|
|[#5798](https://github.com/NVIDIA/spark-rapids/pull/5798)|Fall back to CPU for RoundCeil and RoundFloor expressions|
Expand Down
3 changes: 2 additions & 1 deletion build/buildall
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,6 @@ case $DIST_PROFILE in
320
321
322
330
)
;;

Expand All @@ -171,6 +170,7 @@ case $DIST_PROFILE in
313
320
321
330
)
;;

Expand All @@ -179,6 +179,7 @@ case $DIST_PROFILE in
321cdh
312
320
330
)
;;

Expand Down
10 changes: 5 additions & 5 deletions dist/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -46,12 +46,12 @@
313,
320,
321,
321cdh
321cdh,
330
</noSnapshot.buildvers>
<snapshot.buildvers>
314,
322,
330
322
</snapshot.buildvers>
<databricks.buildvers>
312db,
Expand Down Expand Up @@ -91,14 +91,14 @@
- a downstream distro
- a snapshot
for testing while minimizing the build time
TODO add 3.3.0-SNAPSHOT
-->
<id>minimumFeatureVersionMix</id>
<properties>
<included_buildvers>
312,
320,
321cdh
321cdh,
330
</included_buildvers>
</properties>
</profile>
Expand Down
1 change: 1 addition & 0 deletions docs/additional-functionality/rapids-shuffle.md
Original file line number Diff line number Diff line change
Expand Up @@ -286,6 +286,7 @@ In this section, we are using a docker container built using the sample dockerfi
| 3.2.0 | com.nvidia.spark.rapids.spark320.RapidsShuffleManager |
| 3.2.1 | com.nvidia.spark.rapids.spark321.RapidsShuffleManager |
| 3.2.1 CDH | com.nvidia.spark.rapids.spark321cdh.RapidsShuffleManager |
| 3.3.0 | com.nvidia.spark.rapids.spark330.RapidsShuffleManager |
| Databricks 9.1 | com.nvidia.spark.rapids.spark312db.RapidsShuffleManager |
| Databricks 10.4 | com.nvidia.spark.rapids.spark321db.RapidsShuffleManager |

Expand Down
2 changes: 1 addition & 1 deletion pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -943,7 +943,7 @@
<spark321cdh.version>3.2.1.3.2.7171000.0-3</spark321cdh.version>
<spark321db.version>3.2.1-databricks</spark321db.version>
<spark322.version>3.2.2-SNAPSHOT</spark322.version>
<spark330.version>3.3.1-SNAPSHOT</spark330.version>
<spark330.version>3.3.0</spark330.version>
<spark340.version>3.4.0-SNAPSHOT</spark340.version>
<mockito.version>3.6.0</mockito.version>
<scala.plugin.version>4.3.0</scala.plugin.version>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,7 @@ import com.nvidia.spark.rapids.SparkShimVersion

object SparkShimServiceProvider {
val VERSION = SparkShimVersion(3, 3, 0)
// temporarily allow 3.3.1 while 3.3.0 release candidates are being produced
private val VERSION331 = SparkShimVersion(3, 3, 1)
val VERSIONNAMES = Seq(VERSION, VERSION331).flatMap(v => Seq(s"$v", s"$v-SNAPSHOT"))
val VERSIONNAMES = Seq(s"$VERSION", s"$VERSION-SNAPSHOT")
}

class SparkShimServiceProvider extends com.nvidia.spark.rapids.SparkShimServiceProvider {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,7 @@ import org.apache.spark.sql.types.{DayTimeIntervalType, YearMonthIntervalType}

class Spark330ShimsSuite extends FunSuite {
test("spark shims version") {
// temporarily allow 3.3.1 while 3.3.0 release candidates are being produced
assert(SparkShimImpl.getSparkShimVersion === SparkShimVersion(3, 3, 0) ||
SparkShimImpl.getSparkShimVersion === SparkShimVersion(3, 3, 1))
assert(SparkShimImpl.getSparkShimVersion === SparkShimVersion(3, 3, 0))
}

test("shuffle manager class") {
Expand Down