Skip to content

Commit

Permalink
Merge branch 'branch-22.08' into rapids-5633
Browse files Browse the repository at this point in the history
  • Loading branch information
amahussein committed Jul 22, 2022
2 parents f4db01e + f2d6157 commit 0858a7d
Show file tree
Hide file tree
Showing 28 changed files with 558 additions and 209 deletions.
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,8 +86,8 @@ There is a build script `build/buildall` that automates the local build process.
`./buid/buildall --help` for up-to-date use information.

By default, it builds everything that is needed to create a distribution jar for all released (noSnapshots) Spark versions except for Databricks. Other profiles that you can pass using `--profile=<distribution profile>` include
- `snapshots`
- `minimumFeatureVersionMix` that currently includes 321cdh, 312, 320 is recommended for catching incompatibilities already in the local development cycle
- `snapshots` that includes all released (noSnapshots) and snapshots Spark versions except for Databricks
- `minimumFeatureVersionMix` that currently includes 321cdh, 312, 320, 330 is recommended for catching incompatibilities already in the local development cycle

For initial quick iterations we can use `--profile=<buildver>` to build a single-shim version. e.g., `--profile=311` for Spark 3.1.1.

Expand Down
2 changes: 2 additions & 0 deletions build/buildall
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,7 @@ case $DIST_PROFILE in
320
321
322
330
331
)
;;
Expand All @@ -171,6 +172,7 @@ case $DIST_PROFILE in
313
320
321
322
330
)
;;
Expand Down
2 changes: 1 addition & 1 deletion dist/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -47,11 +47,11 @@
320,
321,
321cdh,
322,
330
</noSnapshot.buildvers>
<snapshot.buildvers>
314,
322,
331
</snapshot.buildvers>
<databricks.buildvers>
Expand Down
1 change: 1 addition & 0 deletions docs/additional-functionality/rapids-shuffle.md
Original file line number Diff line number Diff line change
Expand Up @@ -286,6 +286,7 @@ In this section, we are using a docker container built using the sample dockerfi
| 3.2.0 | com.nvidia.spark.rapids.spark320.RapidsShuffleManager |
| 3.2.1 | com.nvidia.spark.rapids.spark321.RapidsShuffleManager |
| 3.2.1 CDH | com.nvidia.spark.rapids.spark321cdh.RapidsShuffleManager |
| 3.2.2 | com.nvidia.spark.rapids.spark322.RapidsShuffleManager |
| 3.3.0 | com.nvidia.spark.rapids.spark330.RapidsShuffleManager |
| Databricks 9.1 | com.nvidia.spark.rapids.spark312db.RapidsShuffleManager |
| Databricks 10.4 | com.nvidia.spark.rapids.spark321db.RapidsShuffleManager |
Expand Down
1 change: 1 addition & 0 deletions docs/configs.md
Original file line number Diff line number Diff line change
Expand Up @@ -269,6 +269,7 @@ Name | SQL Function(s) | Description | Default Value | Notes
<a name="sql.expression.NaNvl"></a>spark.rapids.sql.expression.NaNvl|`nanvl`|Evaluates to `left` iff left is not NaN, `right` otherwise|true|None|
<a name="sql.expression.NamedLambdaVariable"></a>spark.rapids.sql.expression.NamedLambdaVariable| |A parameter to a higher order SQL function|true|None|
<a name="sql.expression.Not"></a>spark.rapids.sql.expression.Not|`!`, `not`|Boolean not operator|true|None|
<a name="sql.expression.NthValue"></a>spark.rapids.sql.expression.NthValue|`nth_value`|nth window operator|true|None|
<a name="sql.expression.OctetLength"></a>spark.rapids.sql.expression.OctetLength|`octet_length`|The byte length of string data|true|None|
<a name="sql.expression.Or"></a>spark.rapids.sql.expression.Or|`or`|Logical OR|true|None|
<a name="sql.expression.PercentRank"></a>spark.rapids.sql.expression.PercentRank|`percent_rank`|Window function that returns the percent rank value within the aggregation window|true|None|
Expand Down
Loading

0 comments on commit 0858a7d

Please sign in to comment.