Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] cast timezone-awareness check positive for date/time-unrelated types #6138

Closed
gerashegalov opened this issue Jul 28, 2022 · 0 comments · Fixed by #7894
Closed

[BUG] cast timezone-awareness check positive for date/time-unrelated types #6138

gerashegalov opened this issue Jul 28, 2022 · 0 comments · Fixed by #7894
Assignees
Labels
bug Something isn't working

Comments

@gerashegalov
Copy link
Collaborator

Describe the bug
Cast falls back on CPU even if data types are not related to time

 !Expression <Cast> cast(a#2 as double) cannot run on GPU because Only UTC zone id is supported. Actual default zone id: GMT-01:00; Only UTC zone id is supported. Actual session local zone id: GMT-01:00

Steps/Code to reproduce bug

Start Spark in a non-UTC timezone

TZ=UTC+1 ~/dist/spark-3.3.0-bin-hadoop3/bin/pyspark \
  --jars ~/gits/NVIDIA/spark-rapids/dist/target/rapids-4-spark_2.12-22.08.0-SNAPSHOT-cuda11.jar \
  --conf spark.plugins=com.nvidia.spark.SQLPlugin 

Execute:

>>> spark.createDataFrame([(1,)], 'a int').selectExpr('CAST(a AS DOUBLE)').collect() 
22/07/28 16:12:30 WARN GpuOverrides: 
!Exec <ProjectExec> cannot run on GPU because not all expressions can be replaced
  @Expression <Alias> cast(a#2 as double) AS a#4 could run on GPU
    !Expression <Cast> cast(a#2 as double) cannot run on GPU because Only UTC zone id is supported. Actual default zone id: GMT-01:00; Only UTC zone id is supported. Actual session local zone id: GMT-01:00
      @Expression <AttributeReference> a#2 could run on GPU
  ! <RDDScanExec> cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.execution.RDDScanExec
    @Expression <AttributeReference> a#2 could run on GPU

[Row(a=1.0)]

Expected behavior
should be able to run cast such as int to double on GPU regardless of the timezone

Environment details (please complete the following information)

  • Environment location: Any
  • Spark configuration settings related to the issue: N/A

Additional context
#6103

@gerashegalov gerashegalov added bug Something isn't working ? - Needs Triage Need team to review and classify labels Jul 28, 2022
@sameerz sameerz removed the ? - Needs Triage Need team to review and classify label Aug 2, 2022
@firestarman firestarman self-assigned this Mar 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants