Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOC] Misc Doc improvements #4319

Closed
tgravescs opened this issue Dec 7, 2021 · 6 comments
Closed

[DOC] Misc Doc improvements #4319

tgravescs opened this issue Dec 7, 2021 · 6 comments
Assignees
Labels
documentation Improvements or additions to documentation

Comments

@tgravescs
Copy link
Collaborator

tgravescs commented Dec 7, 2021

Report incorrect documentation

Location of incorrect documentation

docs/additional-functionality/rapids-shuffle.md file:

References versions that are Spark snapshot versions, since we don't release them we should remove them from the table.

download.md file:

The plugin is tested on the following architectures mention V100, A100, etc

maybe a nit and again fixed for later, but aren't the "architectures" really Ampere, Volta, Turing vs GPU models?

Requirements mention Apache Hadoop 2.10+ or 3.1.1+ (3.1.1 for nvidia-docker version 2)

do we want to qualify this as its not really required if not using YARN and really these are only required if using docker/cgroups?

Version info mentioned Azure Synapse is 21.10 but not 21.12:

we put Azure Synapse here, but they are in progress of upgrading did we want to put something for Azure and EMR that say see their pages for current version?

@tgravescs tgravescs added documentation Improvements or additions to documentation ? - Needs Triage Need team to review and classify labels Dec 7, 2021
@Salonijain27 Salonijain27 removed the ? - Needs Triage Need team to review and classify label Dec 7, 2021
@nvliyuan nvliyuan self-assigned this Dec 14, 2021
@viadea
Copy link
Collaborator

viadea commented Jan 7, 2022

@tgravescs @nvliyuan
I submitted #4472 for this to address item:

  1. docs/additional-functionality/rapids-shuffle.md
  2. The plugin is tested on the following architectures mention V100, A100, etc

For item:
3. Requirements mention Apache Hadoop 2.10+ or 3.1.1+ (3.1.1 for nvidia-docker version 2)
What is our suggestion for supported Hadoop version?

  1. Version info mentioned Azure Synapse is 21.10 but not 21.12.
    Currently the Azure Synapse only ships with 21.10. I talked with them and once 21.12 is updated officially, we need to remove it from 21.10 and add it to 21.12 download page.

@tgravescs
Copy link
Collaborator Author

for 3. we should either quantify it saying, for yarn deployments that want to use GPU scheduling requires Hadoop 2.10+.... Or alternatively we could remove it form here and say see the documentation for any requirements based on the deployment used.

  1. do we really want to have to keep updating our docs based on this or could we just point to some Azure docs?

@viadea
Copy link
Collaborator

viadea commented Jan 7, 2022

For 4, currently there is no Azure Doc mentioning the version of the plugin. Maybe once it is there, we can point it to Azure doc. However if we do this for Azure Synapse, we may need to do this for EMR as well?

@tgravescs
Copy link
Collaborator Author

it might be a call for Sameer or Karthik, but to me it would be nice just to point to any third party docs since we don't control the versions.

@viadea
Copy link
Collaborator

viadea commented Jan 12, 2022

For #3 #4512 is filed.
For #4, i will update it once Azure Synapse release new version of jars later.

@viadea
Copy link
Collaborator

viadea commented Jan 14, 2022

I think we can close this one now.

@viadea viadea closed this as completed Jan 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

No branches or pull requests

5 participants