Skip to content

Commit

Permalink
Init version 23.10.0-SNAPSHOT (#8796)
Browse files Browse the repository at this point in the history
Bump up project version to 23.10.0-SNAPSHOT in branch-23.10

also include TODO to remind us bump up deps (JNI, private, cudf-py) versions when available #8794

Signed-off-by: Tim Liu <timl@nvidia.com>
  • Loading branch information
NvTimLiu authored Jul 26, 2023
1 parent 4f93896 commit a9b513f
Show file tree
Hide file tree
Showing 28 changed files with 60 additions and 57 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ as a `provided` dependency.
<dependency>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark_2.12</artifactId>
<version>23.08.0-SNAPSHOT</version>
<version>23.10.0-SNAPSHOT</version>
<scope>provided</scope>
</dependency>
```
4 changes: 2 additions & 2 deletions aggregator/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,12 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-parent</artifactId>
<version>23.08.0-SNAPSHOT</version>
<version>23.10.0-SNAPSHOT</version>
</parent>
<artifactId>rapids-4-spark-aggregator_2.12</artifactId>
<name>RAPIDS Accelerator for Apache Spark Aggregator</name>
<description>Creates an aggregated shaded package of the RAPIDS plugin for Apache Spark</description>
<version>23.08.0-SNAPSHOT</version>
<version>23.10.0-SNAPSHOT</version>

<properties>
<!--
Expand Down
4 changes: 2 additions & 2 deletions api_validation/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,10 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-parent</artifactId>
<version>23.08.0-SNAPSHOT</version>
<version>23.10.0-SNAPSHOT</version>
</parent>
<artifactId>rapids-4-spark-api-validation</artifactId>
<version>23.08.0-SNAPSHOT</version>
<version>23.10.0-SNAPSHOT</version>

<profiles>
<profile>
Expand Down
4 changes: 2 additions & 2 deletions delta-lake/delta-20x/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,14 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-parent</artifactId>
<version>23.08.0-SNAPSHOT</version>
<version>23.10.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

<artifactId>rapids-4-spark-delta-20x_2.12</artifactId>
<name>RAPIDS Accelerator for Apache Spark Delta Lake 2.0.x Support</name>
<description>Delta Lake 2.0.x support for the RAPIDS Accelerator for Apache Spark</description>
<version>23.08.0-SNAPSHOT</version>
<version>23.10.0-SNAPSHOT</version>

<properties>
<rapids.compressed.artifact>false</rapids.compressed.artifact>
Expand Down
4 changes: 2 additions & 2 deletions delta-lake/delta-21x/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,14 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-parent</artifactId>
<version>23.08.0-SNAPSHOT</version>
<version>23.10.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

<artifactId>rapids-4-spark-delta-21x_2.12</artifactId>
<name>RAPIDS Accelerator for Apache Spark Delta Lake 2.1.x Support</name>
<description>Delta Lake 2.1.x support for the RAPIDS Accelerator for Apache Spark</description>
<version>23.08.0-SNAPSHOT</version>
<version>23.10.0-SNAPSHOT</version>

<properties>
<rapids.compressed.artifact>false</rapids.compressed.artifact>
Expand Down
4 changes: 2 additions & 2 deletions delta-lake/delta-22x/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,14 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-parent</artifactId>
<version>23.08.0-SNAPSHOT</version>
<version>23.10.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

<artifactId>rapids-4-spark-delta-22x_2.12</artifactId>
<name>RAPIDS Accelerator for Apache Spark Delta Lake 2.2.x Support</name>
<description>Delta Lake 2.2.x support for the RAPIDS Accelerator for Apache Spark</description>
<version>23.08.0-SNAPSHOT</version>
<version>23.10.0-SNAPSHOT</version>

<properties>
<rapids.compressed.artifact>false</rapids.compressed.artifact>
Expand Down
4 changes: 2 additions & 2 deletions delta-lake/delta-24x/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,14 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-parent</artifactId>
<version>23.08.0-SNAPSHOT</version>
<version>23.10.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

<artifactId>rapids-4-spark-delta-24x_2.12</artifactId>
<name>RAPIDS Accelerator for Apache Spark Delta Lake 2.4.x Support</name>
<description>Delta Lake 2.4.x support for the RAPIDS Accelerator for Apache Spark</description>
<version>23.08.0-SNAPSHOT</version>
<version>23.10.0-SNAPSHOT</version>

<properties>
<rapids.compressed.artifact>false</rapids.compressed.artifact>
Expand Down
4 changes: 2 additions & 2 deletions delta-lake/delta-spark321db/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,14 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-parent</artifactId>
<version>23.08.0-SNAPSHOT</version>
<version>23.10.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

<artifactId>rapids-4-spark-delta-spark321db_2.12</artifactId>
<name>RAPIDS Accelerator for Apache Spark Databricks 10.4 Delta Lake Support</name>
<description>Databricks 10.4 Delta Lake support for the RAPIDS Accelerator for Apache Spark</description>
<version>23.08.0-SNAPSHOT</version>
<version>23.10.0-SNAPSHOT</version>

<properties>
<rapids.compressed.artifact>false</rapids.compressed.artifact>
Expand Down
4 changes: 2 additions & 2 deletions delta-lake/delta-spark330db/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,14 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-parent</artifactId>
<version>23.08.0-SNAPSHOT</version>
<version>23.10.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

<artifactId>rapids-4-spark-delta-spark330db_2.12</artifactId>
<name>RAPIDS Accelerator for Apache Spark Databricks 11.3 Delta Lake Support</name>
<description>Databricks 11.3 Delta Lake support for the RAPIDS Accelerator for Apache Spark</description>
<version>23.08.0-SNAPSHOT</version>
<version>23.10.0-SNAPSHOT</version>

<properties>
<rapids.compressed.artifact>false</rapids.compressed.artifact>
Expand Down
4 changes: 2 additions & 2 deletions delta-lake/delta-spark332db/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,14 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-parent</artifactId>
<version>23.08.0-SNAPSHOT</version>
<version>23.10.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

<artifactId>rapids-4-spark-delta-spark332db_2.12</artifactId>
<name>RAPIDS Accelerator for Apache Spark Databricks 12.2 Delta Lake Support</name>
<description>Databricks 12.2 Delta Lake support for the RAPIDS Accelerator for Apache Spark</description>
<version>23.08.0-SNAPSHOT</version>
<version>23.10.0-SNAPSHOT</version>

<properties>
<rapids.compressed.artifact>false</rapids.compressed.artifact>
Expand Down
4 changes: 2 additions & 2 deletions delta-lake/delta-stub/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,14 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-parent</artifactId>
<version>23.08.0-SNAPSHOT</version>
<version>23.10.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

<artifactId>rapids-4-spark-delta-stub_2.12</artifactId>
<name>RAPIDS Accelerator for Apache Spark Delta Lake Stub</name>
<description>Delta Lake stub for the RAPIDS Accelerator for Apache Spark</description>
<version>23.08.0-SNAPSHOT</version>
<version>23.10.0-SNAPSHOT</version>

<properties>
<rapids.compressed.artifact>false</rapids.compressed.artifact>
Expand Down
4 changes: 2 additions & 2 deletions dist/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,12 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-parent</artifactId>
<version>23.08.0-SNAPSHOT</version>
<version>23.10.0-SNAPSHOT</version>
</parent>
<artifactId>rapids-4-spark_2.12</artifactId>
<name>RAPIDS Accelerator for Apache Spark Distribution</name>
<description>Creates the distribution package of the RAPIDS plugin for Apache Spark</description>
<version>23.08.0-SNAPSHOT</version>
<version>23.10.0-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>com.nvidia</groupId>
Expand Down
2 changes: 1 addition & 1 deletion docs/configs.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ The following is the list of options that `rapids-plugin-4-spark` supports.
On startup use: `--conf [conf key]=[conf value]`. For example:

```
${SPARK_HOME}/bin/spark-shell --jars rapids-4-spark_2.12-23.08.0-SNAPSHOT-cuda11.jar \
${SPARK_HOME}/bin/spark-shell --jars rapids-4-spark_2.12-23.10.0-SNAPSHOT-cuda11.jar \
--conf spark.plugins=com.nvidia.spark.SQLPlugin \
--conf spark.rapids.sql.concurrentGpuTasks=2
```
Expand Down
12 changes: 6 additions & 6 deletions docs/dev/shims.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,17 +68,17 @@ Using JarURLConnection URLs we create a Parallel World of the current version wi
Spark 3.0.2's URLs:

```text
jar:file:/home/spark/rapids-4-spark_2.12-23.08.0.jar!/
jar:file:/home/spark/rapids-4-spark_2.12-23.08.0.jar!/spark3xx-common/
jar:file:/home/spark/rapids-4-spark_2.12-23.08.0.jar!/spark302/
jar:file:/home/spark/rapids-4-spark_2.12-23.10.0.jar!/
jar:file:/home/spark/rapids-4-spark_2.12-23.10.0.jar!/spark3xx-common/
jar:file:/home/spark/rapids-4-spark_2.12-23.10.0.jar!/spark302/
```

Spark 3.2.0's URLs :

```text
jar:file:/home/spark/rapids-4-spark_2.12-23.08.0.jar!/
jar:file:/home/spark/rapids-4-spark_2.12-23.08.0.jar!/spark3xx-common/
jar:file:/home/spark/rapids-4-spark_2.12-23.08.0.jar!/spark320/
jar:file:/home/spark/rapids-4-spark_2.12-23.10.0.jar!/
jar:file:/home/spark/rapids-4-spark_2.12-23.10.0.jar!/spark3xx-common/
jar:file:/home/spark/rapids-4-spark_2.12-23.10.0.jar!/spark320/
```

### Late Inheritance in Public Classes
Expand Down
4 changes: 2 additions & 2 deletions docs/get-started/getting-started-on-prem.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,13 +56,13 @@ CUDA and will not run on other versions. The jars use a classifier to keep them
- CUDA 11.x => classifier cuda11

For example, here is a sample version of the jar with CUDA 11.x support:
- rapids-4-spark_2.12-23.08.0-SNAPSHOT-cuda11.jar
- rapids-4-spark_2.12-23.10.0-SNAPSHOT-cuda11.jar

For simplicity export the location to this jar. This example assumes the sample jar above has
been placed in the `/opt/sparkRapidsPlugin` directory:
```shell
export SPARK_RAPIDS_DIR=/opt/sparkRapidsPlugin
export SPARK_RAPIDS_PLUGIN_JAR=${SPARK_RAPIDS_DIR}/rapids-4-spark_2.12-23.08.0-SNAPSHOT-cuda11.jar
export SPARK_RAPIDS_PLUGIN_JAR=${SPARK_RAPIDS_DIR}/rapids-4-spark_2.12-23.10.0-SNAPSHOT-cuda11.jar
```

## Install the GPU Discovery Script
Expand Down
6 changes: 3 additions & 3 deletions integration_tests/DATA_GEN.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,12 @@ corresponding profile flag `-P<jdk11|jdk17>`

After this the jar should be at
`target/rapids-4-spark-integration-tests_2.12-$PLUGIN_VERSION-spark$SPARK_VERSION.jar`
for example a Spark 3.3.0 jar for the 23.08.0 release would be
`target/rapids-4-spark-integration-tests_2.12-23.08.0-spark330.jar`
for example a Spark 3.3.0 jar for the 23.10.0 release would be
`target/rapids-4-spark-integration-tests_2.12-23.10.0-spark330.jar`

To get a spark shell with this you can run
```shell
spark-shell --jars target/rapids-4-spark-integration-tests_2.12-23.08.0-spark330.jar
spark-shell --jars target/rapids-4-spark-integration-tests_2.12-23.10.0-spark330.jar
```

After that you should be good to go.
Expand Down
6 changes: 3 additions & 3 deletions integration_tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -250,7 +250,7 @@ individually, so you don't risk running unit tests along with the integration te
http://www.scalatest.org/user_guide/using_the_scalatest_shell

```shell
spark-shell --jars rapids-4-spark-tests_2.12-23.08.0-SNAPSHOT-tests.jar,rapids-4-spark-integration-tests_2.12-23.08.0-SNAPSHOT-tests.jar,scalatest_2.12-3.0.5.jar,scalactic_2.12-3.0.5.jar
spark-shell --jars rapids-4-spark-tests_2.12-23.10.0-SNAPSHOT-tests.jar,rapids-4-spark-integration-tests_2.12-23.10.0-SNAPSHOT-tests.jar,scalatest_2.12-3.0.5.jar,scalactic_2.12-3.0.5.jar
```

First you import the `scalatest_shell` and tell the tests where they can find the test files you
Expand All @@ -273,7 +273,7 @@ If you just want to verify the SQL replacement is working you will need to add t
assumes CUDA 11.0 is being used.

```
$SPARK_HOME/bin/spark-submit --jars "rapids-4-spark_2.12-23.08.0-SNAPSHOT-cuda11.jar" ./runtests.py
$SPARK_HOME/bin/spark-submit --jars "rapids-4-spark_2.12-23.10.0-SNAPSHOT-cuda11.jar" ./runtests.py
```

You don't have to enable the plugin for this to work, the test framework will do that for you.
Expand Down Expand Up @@ -372,7 +372,7 @@ To run cudf_udf tests, need following configuration changes:
As an example, here is the `spark-submit` command with the cudf_udf parameter on CUDA 11.0:

```
$SPARK_HOME/bin/spark-submit --jars "rapids-4-spark_2.12-23.08.0-SNAPSHOT-cuda11.jar,rapids-4-spark-tests_2.12-23.08.0-SNAPSHOT.jar" --conf spark.rapids.memory.gpu.allocFraction=0.3 --conf spark.rapids.python.memory.gpu.allocFraction=0.3 --conf spark.rapids.python.concurrentPythonWorkers=2 --py-files "rapids-4-spark_2.12-23.08.0-SNAPSHOT-cuda11.jar" --conf spark.executorEnv.PYTHONPATH="rapids-4-spark_2.12-23.08.0-SNAPSHOT-cuda11.jar" ./runtests.py --cudf_udf
$SPARK_HOME/bin/spark-submit --jars "rapids-4-spark_2.12-23.10.0-SNAPSHOT-cuda11.jar,rapids-4-spark-tests_2.12-23.10.0-SNAPSHOT.jar" --conf spark.rapids.memory.gpu.allocFraction=0.3 --conf spark.rapids.python.memory.gpu.allocFraction=0.3 --conf spark.rapids.python.concurrentPythonWorkers=2 --py-files "rapids-4-spark_2.12-23.10.0-SNAPSHOT-cuda11.jar" --conf spark.executorEnv.PYTHONPATH="rapids-4-spark_2.12-23.10.0-SNAPSHOT-cuda11.jar" ./runtests.py --cudf_udf
```

### Enabling fuzz tests
Expand Down
4 changes: 2 additions & 2 deletions integration_tests/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,10 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-parent</artifactId>
<version>23.08.0-SNAPSHOT</version>
<version>23.10.0-SNAPSHOT</version>
</parent>
<artifactId>rapids-4-spark-integration-tests_2.12</artifactId>
<version>23.08.0-SNAPSHOT</version>
<version>23.10.0-SNAPSHOT</version>
<properties>
<target.classifier/>
<rapids.default.jar.excludePattern>**/*</rapids.default.jar.excludePattern>
Expand Down
2 changes: 1 addition & 1 deletion jenkins/databricks/create.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ def main():
workspace = 'https://dbc-9ff9942e-a9c4.cloud.databricks.com'
token = ''
sshkey = ''
cluster_name = 'CI-GPU-databricks-23.08.0-SNAPSHOT'
cluster_name = 'CI-GPU-databricks-23.10.0-SNAPSHOT'
idletime = 240
runtime = '7.0.x-gpu-ml-scala2.12'
num_workers = 1
Expand Down
1 change: 1 addition & 0 deletions jenkins/databricks/init_cudf_udf.sh
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@

set -ex

# TODO: https://github.com/NVIDIA/spark-rapids/issues/8794
CUDF_VER=${CUDF_VER:-23.08}
CUDA_VER=${CUDA_VER:-11.8.0}

Expand Down
5 changes: 3 additions & 2 deletions jenkins/version-def.sh
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,11 @@ for VAR in $OVERWRITE_PARAMS; do
done
IFS=$PRE_IFS

# TODO: https://github.com/NVIDIA/spark-rapids/issues/8794
CUDF_VER=${CUDF_VER:-"23.08.0-SNAPSHOT"}
CUDA_CLASSIFIER=${CUDA_CLASSIFIER:-"cuda11"}
PROJECT_VER=${PROJECT_VER:-"23.08.0-SNAPSHOT"}
PROJECT_TEST_VER=${PROJECT_TEST_VER:-"23.08.0-SNAPSHOT"}
PROJECT_VER=${PROJECT_VER:-"23.10.0-SNAPSHOT"}
PROJECT_TEST_VER=${PROJECT_TEST_VER:-"23.10.0-SNAPSHOT"}
SPARK_VER=${SPARK_VER:-"3.1.1"}
# Make a best attempt to set the default value for the shuffle shim.
# Note that SPARK_VER for non-Apache Spark flavors (i.e. databricks,
Expand Down
3 changes: 2 additions & 1 deletion pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
<artifactId>rapids-4-spark-parent</artifactId>
<name>RAPIDS Accelerator for Apache Spark Root Project</name>
<description>The root project of the RAPIDS Accelerator for Apache Spark</description>
<version>23.08.0-SNAPSHOT</version>
<version>23.10.0-SNAPSHOT</version>
<packaging>pom</packaging>

<url>https://nvidia.github.io/spark-rapids/</url>
Expand Down Expand Up @@ -548,6 +548,7 @@
<parquet.hadoop.version>1.10.1</parquet.hadoop.version>
<spark.version.classifier>spark${buildver}</spark.version.classifier>
<cuda.version>cuda11</cuda.version>
<!-- TODO: https://github.com/NVIDIA/spark-rapids/issues/8794 -->
<spark-rapids-jni.version>23.08.0-SNAPSHOT</spark-rapids-jni.version>
<spark-rapids-private.version>23.08.0-SNAPSHOT</spark-rapids-private.version>
<scala.binary.version>2.12</scala.binary.version>
Expand Down
4 changes: 2 additions & 2 deletions shuffle-plugin/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,13 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-parent</artifactId>
<version>23.08.0-SNAPSHOT</version>
<version>23.10.0-SNAPSHOT</version>
</parent>

<artifactId>rapids-4-spark-shuffle_2.12</artifactId>
<name>RAPIDS Accelerator for Apache Spark Shuffle Plugin</name>
<description>Accelerated shuffle plugin for the RAPIDS plugin for Apache Spark</description>
<version>23.08.0-SNAPSHOT</version>
<version>23.10.0-SNAPSHOT</version>

<properties>
<rapids.compressed.artifact>false</rapids.compressed.artifact>
Expand Down
4 changes: 2 additions & 2 deletions sql-plugin/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,12 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-parent</artifactId>
<version>23.08.0-SNAPSHOT</version>
<version>23.10.0-SNAPSHOT</version>
</parent>
<artifactId>rapids-4-spark-sql_2.12</artifactId>
<name>RAPIDS Accelerator for Apache Spark SQL Plugin</name>
<description>The RAPIDS SQL plugin for Apache Spark</description>
<version>23.08.0-SNAPSHOT</version>
<version>23.10.0-SNAPSHOT</version>

<properties>
<rapids.compressed.artifact>false</rapids.compressed.artifact>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1996,7 +1996,7 @@ object RapidsConf {
|On startup use: `--conf [conf key]=[conf value]`. For example:
|
|```
|${SPARK_HOME}/bin/spark-shell --jars rapids-4-spark_2.12-23.08.0-SNAPSHOT-cuda11.jar \
|${SPARK_HOME}/bin/spark-shell --jars rapids-4-spark_2.12-23.10.0-SNAPSHOT-cuda11.jar \
|--conf spark.plugins=com.nvidia.spark.SQLPlugin \
|--conf spark.rapids.sql.concurrentGpuTasks=2
|```
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -58,13 +58,13 @@ import org.apache.spark.util.MutableURLClassLoader
E.g., Spark 3.2.0 Shim will use
jar:file:/home/spark/rapids-4-spark_2.12-23.08.0.jar!/spark3xx-common/
jar:file:/home/spark/rapids-4-spark_2.12-23.08.0.jar!/spark320/
jar:file:/home/spark/rapids-4-spark_2.12-23.10.0.jar!/spark3xx-common/
jar:file:/home/spark/rapids-4-spark_2.12-23.10.0.jar!/spark320/
Spark 3.1.1 will use
jar:file:/home/spark/rapids-4-spark_2.12-23.08.0.jar!/spark3xx-common/
jar:file:/home/spark/rapids-4-spark_2.12-23.08.0.jar!/spark311/
jar:file:/home/spark/rapids-4-spark_2.12-23.10.0.jar!/spark3xx-common/
jar:file:/home/spark/rapids-4-spark_2.12-23.10.0.jar!/spark311/
Using these Jar URL's allows referencing different bytecode produced from identical sources
by incompatible Scala / Spark dependencies.
Expand Down
Loading

0 comments on commit a9b513f

Please sign in to comment.