Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Use double quotes for SQL configs in error messages #5750

Open
nartal1 opened this issue Jun 6, 2022 · 1 comment
Open

[FEA] Use double quotes for SQL configs in error messages #5750

nartal1 opened this issue Jun 6, 2022 · 1 comment
Labels
audit_3.3.0 Audit related tasks for 3.3.0 audit_3.4.0 Audit related tasks for 3.4.0 feature request New feature or request good first issue Good for newcomers

Comments

@nartal1
Copy link
Collaborator

nartal1 commented Jun 6, 2022

Is your feature request related to a problem? Please describe.
This is a minor change in Spark where double quotes is wrapped for types and config keys to easily parse them from error messages.
We need to audit our code to check all the methods which we call from QueryExecutionErrors and make necessary changes. One of the examples is :

Plugin's error message:
org.apache.spark.SparkDateTimeException: The value NaN of the type DOUBLE cannot be cast to TIMESTAMP because it is malformed. Correct the value as per the syntax, or change its target type. To return NULL instead, use try_cast. If necessary set spark.sql.ansi.enabled to "false" to bypass this error.

Spark's error message:
org.apache.spark.SparkDateTimeException: The value NaN of the type "DOUBLE" cannot be cast to "TIMESTAMP" because it is malformed. Correct the value as per the syntax, or change its target type. To return NULL instead, use try_cast. If necessary set "spark.sql.ansi.enabled" to "false" to bypass this error.

Spark commits:

  1. apache/spark@2e102b8bd2
  2. apache/spark@5dfa24d154
  3. apache/spark@f0564354f1
  4. apache/spark@e088c820e1
@nartal1 nartal1 added feature request New feature or request ? - Needs Triage Need team to review and classify audit_3.3.0 Audit related tasks for 3.3.0 audit_3.4.0 Audit related tasks for 3.4.0 labels Jun 6, 2022
@sameerz sameerz added good first issue Good for newcomers and removed ? - Needs Triage Need team to review and classify labels Jun 7, 2022
@sameerz
Copy link
Collaborator

sameerz commented Jun 7, 2022

Related to #4210

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
audit_3.3.0 Audit related tasks for 3.3.0 audit_3.4.0 Audit related tasks for 3.4.0 feature request New feature or request good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

2 participants