Fix udf-compiler scala2.13 internal return statements #11553
Conversation
Signed-off-by: Alessandro Bellina <abellina@nvidia.com>
abellina
requested review from
jlowe,
revans2,
tgravescs,
GaryShen2008,
NvTimLiu and
gerashegalov
as code owners
|
@seanprime7 I made the suggested change, please take another look when you get a chance. |
|
build |
| @@ -36,6 +36,7 @@ class OpcodeSuite extends AnyFunSuite { | |||
| val conf: SparkConf = new SparkConf() | |||
| .set("spark.sql.extensions", "com.nvidia.spark.udf.Plugin") | |||
| .set("spark.rapids.sql.udfCompiler.enabled", "true") | |||
| .set("spark.sql.ansi.enabled", "false") | |||
There was a problem hiding this comment.
Curious why we need to disable ANSI here? I assume this is for Spark 4.0, but then I would expect there to be corresponding negative tests where we are testing that we do not replace UDFs because of ANSI expression semantics. Seems like we need a comment and/or pointer to a tracking issue here.
There was a problem hiding this comment.
correct that is due to spark 4, let me file something at least with the failures I see with ansi enabled.
There was a problem hiding this comment.
Filed #11633 to address as a separate issue. One thing that is nice about this failure is that it is detected at plan time, and we won't cause a corruption.
|
build |
Closes #11174
This is in draft to solicit comments from @seanprime7