[FEA] Use double quotes for SQL configs in error messages #5750
Labels
audit_3.3.0
Audit related tasks for 3.3.0
audit_3.4.0
Audit related tasks for 3.4.0
feature request
New feature or request
good first issue
Good for newcomers
Is your feature request related to a problem? Please describe.
This is a minor change in Spark where double quotes is wrapped for types and config keys to easily parse them from error messages.
We need to audit our code to check all the methods which we call from QueryExecutionErrors and make necessary changes. One of the examples is :
Plugin's error message:
org.apache.spark.SparkDateTimeException: The value NaN of the type DOUBLE cannot be cast to TIMESTAMP because it is malformed. Correct the value as per the syntax, or change its target type. To return NULL instead, use
try_cast. If necessary set spark.sql.ansi.enabled to "false" to bypass this error.
Spark's error message:
org.apache.spark.SparkDateTimeException: The value NaN of the type "DOUBLE" cannot be cast to "TIMESTAMP" because it is malformed. Correct the value as per the syntax, or change its target type. To return NULL instead, use
try_cast. If necessary set "spark.sql.ansi.enabled" to "false" to bypass this error.
Spark commits:
The text was updated successfully, but these errors were encountered: