Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Spark 3.3 integration test failures logic_test.py::test_logical_with_side_effect #5244

Closed
tgravescs opened this issue Apr 13, 2022 · 4 comments
Assignees
Labels
bug Something isn't working P0 Must have for release

Comments

@tgravescs
Copy link
Collaborator

Describe the bug

10:23:26  FAILED ../../src/main/python/logic_test.py::test_logical_with_side_effect[NULL-2147483647-true-AND]
10:23:26  FAILED ../../src/main/python/logic_test.py::test_logical_with_side_effect[NULL-2147483647-true-OR]
10:23:26  FAILED ../../src/main/python/logic_test.py::test_logical_with_side_effect[a-2147483647-true-OR]
10:23:26  FAILED ../../src/main/python/logic_test.py::test_logical_with_side_effect[b-2147483647-true-AND]
10:23:26  ../../src/main/python/logic_test.py::test_logical_with_side_effect[NULL-2147483647-true-AND] 22/04/13 15:23:15 WARN TaskSetManager: Lost task 0.0 in stage 10.0 (TID 20) (executor 0): org.apache.spark.SparkArithmeticException: integer overflow. If necessary set spark.sql.ansi.enabled to false (except for ANSI interval type) to bypass this error.
10:23:26  == SQL(line 1, position 10) ==
10:23:26  NULL AND (c + 2) > 0
10:23:26            ^^^^^
10:23:26  
10:23:26  	at org.apache.spark.sql.errors.QueryExecutionErrors$.arithmeticOverflowError(QueryExecutionErrors.scala:458)
10:23:26  	at org.apache.spark.sql.catalyst.util.MathUtils$.withOverflow(MathUtils.scala:76)
10:23:26  	at org.apache.spark.sql.catalyst.util.MathUtils$.addExact(MathUtils.scala:30)
10:23:26  	at org.apache.spark.sql.catalyst.util.MathUtils.addExact(MathUtils.scala)
10:23:26  	at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown Source)
@tgravescs tgravescs added bug Something isn't working ? - Needs Triage Need team to review and classify P0 Must have for release labels Apr 13, 2022
@andygrove andygrove added this to the Apr 4 - Apr 15 milestone Apr 13, 2022
@andygrove
Copy link
Contributor

The test is expecting java.lang.ArithmeticException but Spark 3.3 now throws a SparkArithmeticException

@sameerz
Copy link
Collaborator

sameerz commented Apr 13, 2022

Is this related to #5196 ?

@andygrove
Copy link
Contributor

Is this related to #5196 ?

Yes, #5196 is the real fix for this. #5247 is a temporary workaround until we implement that.

@andygrove
Copy link
Contributor

Fixed in #5247

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working P0 Must have for release
Projects
None yet
Development

No branches or pull requests

3 participants