Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Drop support for Spark 3.0.1, 3.0.2, 3.0.3, Databricks 7.3 ML LTS #4915

Closed
sameerz opened this issue Mar 8, 2022 · 3 comments
Closed
Assignees
Labels
feature request New feature or request

Comments

@sameerz
Copy link
Collaborator

sameerz commented Mar 8, 2022

Is your feature request related to a problem? Please describe.
N/A

Describe the solution you'd like
We will drop support for older versions of Spark in 22.04. The supported versions for 22.04 will be:

  • Apache Spark 3.1.1, 3.1.2, 3.1.3, 3.2.0, 3.2.1, Cloudera CDP 7.1.6, 7.1.7, Databricks 9.1 ML LTS, 10.4 ML LTS Runtime and GCP Dataproc 2.0

We will support Apache Spark 3.3.0 in 22.04 if it is released before April 15. Otherwise it will be a hotfix for 22.04 (if requested) or supported in 22.06.

For reference the versions supported in 22.02 were here: https://nvidia.github.io/spark-rapids/docs/download.html#release-v22020

Describe alternatives you've considered
N/A

Additional context
We are removing older versions of Spark distributions. In case anyone has a request for support for an older version please comment here.

Update
We will keep support of 3.1.1 in 22.04, and remove it in a future release.

@sameerz sameerz added the feature request New feature or request label Mar 8, 2022
@sameerz sameerz added this to the Feb 28 - Mar 18 milestone Mar 8, 2022
@pxLi
Copy link
Collaborator

pxLi commented Mar 9, 2022

are we going to support spark 3.0.4? or we are dropping support for all spark 30X versions?

@sameerz
Copy link
Collaborator Author

sameerz commented Mar 9, 2022

The plan is to remove support for Spark 3.0.x altogether.

cc: @krajendrannv

@sameerz sameerz changed the title [FEA] Drop support for Spark 3.0.1, 3.0.2, 3.0.3, 3.1.1, Databricks 7.3 ML LTS [FEA] Drop support for Spark 3.0.1, 3.0.2, 3.0.3, Databricks 7.3 ML LTS Mar 18, 2022
@jlowe
Copy link
Member

jlowe commented Mar 22, 2022

Fixed by #4988.

@jlowe jlowe closed this as completed Mar 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants