Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support running GetArrayStructFields on GPU[databricks] #4875

Merged
merged 5 commits into from
Mar 5, 2022

Conversation

firestarman
Copy link
Collaborator

@firestarman firestarman commented Feb 28, 2022

This PR is adding the support of running GetArrayStructFields on GPU, along with the relevant tests.

closes #4816

Signed-off-by: Firestarman firestarmanllc@gmail.com

Signed-off-by: Firestarman <firestarmanllc@gmail.com>
@firestarman firestarman changed the title Support running GetArrayStructFields on GPU Support running GetArrayStructFields on GPU[databricks] Feb 28, 2022
Signed-off-by: Firestarman <firestarmanllc@gmail.com>
@firestarman
Copy link
Collaborator Author

build

Signed-off-by: Firestarman <firestarmanllc@gmail.com>
Signed-off-by: Firestarman <firestarmanllc@gmail.com>
revans2
revans2 previously approved these changes Feb 28, 2022
Copy link
Collaborator

@revans2 revans2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks great I just had one small nit I can live without.

structView.getChildColumnView(ordinal)
}
val listView = withResource(fieldView) { _ =>
new ColumnView(DType.LIST, base.getRowCount, Optional.of[java.lang.Long](base.getNullCount),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Could you use GpuListUtils.replaceListDataColumnAsView instead? It does the same thing, but adds a few more checks, and would make more of this kind of scary code common.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to know this.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

jlowe
jlowe previously approved these changes Feb 28, 2022
@jlowe jlowe added this to the Feb 28 - Mar 18 milestone Feb 28, 2022
@sameerz sameerz added the feature request New feature or request label Feb 28, 2022
Signed-off-by: Firestarman <firestarmanllc@gmail.com>
@firestarman firestarman dismissed stale reviews from jlowe and revans2 via 51f6c43 March 1, 2022 01:16
@firestarman
Copy link
Collaborator Author

build

1 similar comment
@firestarman
Copy link
Collaborator Author

build

@firestarman
Copy link
Collaborator Author

#4881 is to fix the build error in premerge.

@sameerz
Copy link
Collaborator

sameerz commented Mar 4, 2022

build

@firestarman firestarman merged commit 89a3785 into NVIDIA:branch-22.04 Mar 5, 2022
@firestarman firestarman deleted the array_struct branch March 5, 2022 11:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEA] Support org.apache.spark.sql.catalyst.expressions.GetArrayStructFields
4 participants