Skip the test for spark versions 3.1.1, 3.1.2 and 3.2.0 only [databricks]

[razajafri]

The fix was posted for 3.1.3+ and 3.2.1+ which makes the bug valid only in version 3.1.1, 3.1.2 and 3.2.0

Fixes #4488

Signed-off-by: Raza Jafri rjafri@nvidia.com

[jlowe]

I expected to see a change to Spark31XShims to have isCastingStringToNegDecimalScaleSupported return false, but it's not here. I don't see how the spark311 and spark312 shims are picking up the changes to mark them properly.

[tgravescs]

31xShim already has it set to false:
https://github.com/NVIDIA/spark-rapids/blob/branch-22.02/sql-plugin/src/main/311until320-nondb/scala/com/nvidia/spark/rapids/shims/v2/Spark31XShims.scala#L546

I believe you can remove it from Spark330Shims as well now.

[tgravescs]

this does seem a bit dangerous in the sense if we add a databricks 3.1.3 shim we could easily miss this.
perhaps at least put a comment here. I assume the test won't fail if we get it wrong?

[razajafri]

The test isn't skipped for 3.1.3, it will blow up saying part of the plan isn't columnar as there is a check in GpuCast for it if we forget to override it for Spark-3.1.3-db.

I can move this to Spark312dbShims.

[razajafri]

I expected to see a change to Spark31XShims to have isCastingStringToNegDecimalScaleSupported return false, but it's not here. I don't see how the spark311 and spark312 shims are picking up the changes to mark them properly.

It's already there in the Spark31XShims https://github.com/razajafri/spark-rapids-1/blob/9e601d639caafef010e33717a7e46f82dd2c92e5/sql-plugin/src/main/311until320-nondb/scala/com/nvidia/spark/rapids/shims/v2/Spark31XShims.scala#L546. That should cover it?

[razajafri]

build

[razajafri]

build