Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] ansi_cast tests are failing in 3.1.0 #1164

Closed
revans2 opened this issue Nov 19, 2020 · 5 comments · Fixed by #1166
Closed

[BUG] ansi_cast tests are failing in 3.1.0 #1164

revans2 opened this issue Nov 19, 2020 · 5 comments · Fixed by #1166
Assignees
@revans2
Labels
bug Something isn't working P0 Must have for release Spark 3.1+ Bugs only related to Spark 3.1 or higher test Only impacts tests
Milestone
Nov 9 - Nov 20

Comments

@revans2
Copy link
Collaborator

revans2 commented Nov 19, 2020 •
edited
Loading

Describe the bug
When running cast tests against 3.1.0 we now get failures about not being able to cast between numbers and timestamps. We probably just need to enable a flag to keep the tests running in the short term.
@revans2 revans2 added bug Something isn't working test Only impacts tests P0 Must have for release labels Nov 19, 2020
@revans2 revans2 added this to the Nov 9 - Nov 20 milestone Nov 19, 2020
@revans2 revans2 self-assigned this Nov 19, 2020
@revans2 revans2 added invalid This doesn't seem right and removed P0 Must have for release labels Nov 19, 2020
@revans2
Copy link
Collaborator Author

revans2 commented Nov 19, 2020

This failed in CI, but I cannot reproduce this.

@revans2 revans2 closed this as completed Nov 19, 2020
@revans2
Copy link
Collaborator Author

revans2 commented Nov 19, 2020

Actually it is failing consistently in CI. I just cannot reproduce it locally.

@revans2 revans2 reopened this Nov 19, 2020
@revans2 revans2 changed the title [BUG] ansi_cast tests are failing in 3.0.2 [BUG] ansi_cast tests are failing in 3.1.0 Nov 19, 2020
@revans2 revans2 added P0 Must have for release and removed invalid This doesn't seem right labels Nov 19, 2020
@revans2
Copy link
Collaborator Author

revans2 commented Nov 19, 2020

It looks almost like our CI has an invalid version of 3.1.0.

The cast is allowed by default, and has been that way for 5 months.

https://github.com/apache/spark/blob/3695e997d5d436be086235505bbb030c87ae8eef/sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala#L2827-L2834

So I don't really know what is happening here.

@revans2
Copy link
Collaborator Author

revans2 commented Nov 19, 2020

OK I was able to reproduce the error now, but I had to get the version of spark-3.1.0-SNAPSHOT that is stored in the internal CI mirror being used. It looks like it is wrong some how.

@revans2
Copy link
Collaborator Author

revans2 commented Nov 19, 2020

OK I think I have tracked this down. apache/spark#30260 went in recently that makes it so these casts are not supported in ANSI mode. It looks like the real problem is that the error message being reported is no longer correct.

@sameerz sameerz added the Spark 3.1+ Bugs only related to Spark 3.1 or higher label Nov 19, 2020
@revans2 revans2 mentioned this issue Nov 19, 2020
Merged
tgravescs pushed a commit to tgravescs/spark-rapids that referenced this issue Nov 30, 2023
@nvauto
Merge pull request NVIDIA#1164 from NVIDIA/bot-auto-merge-branch-23.06
0f8bd47 
[auto-merge] bot-auto-merge-branch-23.06 to branch-23.08 [skip ci] [bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@revans2 revans2

Labels
bug Something isn't working P0 Must have for release Spark 3.1+ Bugs only related to Spark 3.1 or higher test Only impacts tests
Projects
None yet
Milestone
Nov 9 - Nov 20
Development

Successfully merging a pull request may close this issue.

Fix the cast tests for 3.1.0+ revans2/spark-rapids
2 participants
@revans2 @sameerz