Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEA][Audit] - [SPARK-37822][SQL] StringSplit should return an array on non-null elements #4671

Closed
nartal1 opened this issue Jan 31, 2022 · 2 comments · Fixed by #4716
Closed

[FEA][Audit] - [SPARK-37822][SQL] StringSplit should return an array on non-null elements #4671

nartal1 opened this issue Jan 31, 2022 · 2 comments · Fixed by #4716
Assignees
@HaoYang670
Labels
audit_3.3.0 Audit related tasks for 3.3.0 P1 Nice to have for release

Comments

@nartal1
Copy link
Collaborator

nartal1 commented Jan 31, 2022

Is your feature request related to a problem? Please describe.
Trivial change for accurate return type of StringSplit as there isn't any case where the array can contain nulls. I am not sure if we really need to pull this in.
PR - apache/spark@db15b82be9
@nartal1 nartal1 added feature request New feature or request ? - Needs Triage Need team to review and classify audit_3.3.0 Audit related tasks for 3.3.0 labels Jan 31, 2022
@sameerz sameerz added P1 Nice to have for release and removed feature request New feature or request ? - Needs Triage Need team to review and classify labels Feb 1, 2022
@HaoYang670 HaoYang670 self-assigned this Feb 7, 2022
@HaoYang670
Copy link
Collaborator

HaoYang670 commented Feb 7, 2022 •
edited
Loading

Do we need to edit in the shim layer? The change is trivial and no new feature is introduced.

@revans2
Copy link
Collaborator

revans2 commented Feb 7, 2022

It was a small bug fix and like the logic stated there should be no cases where the null can be returned in the array, so I think we can just make the change and not worry about putting it under a shim.

@HaoYang670 HaoYang670 mentioned this issue Feb 8, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@HaoYang670 HaoYang670

Labels
audit_3.3.0 Audit related tasks for 3.3.0 P1 Nice to have for release
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

GpuStringSplit should return an array on not-null elements HaoYang670/spark-rapids
4 participants
@revans2 @sameerz @nartal1 @HaoYang670