Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Query failure with java.lang.AssertionError when using partitioned Iceberg tables #5464

Closed
mattahrens opened this issue May 11, 2022 · 0 comments · Fixed by #5465
Closed
Assignees
Labels
bug Something isn't working

Comments

@mattahrens
Copy link
Collaborator

mattahrens commented May 11, 2022

Describe the bug
When running the NDS 2.0 Power Run locally using partitioned Iceberg tables with Parquet data, the following exception is encountered when query 7 is run:

Caused by: java.lang.AssertionError: assertion failed
	at scala.Predef$.assert(Predef.scala:208)
	at org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanExec.$anonfun$doExecuteBroadcast$1(AdaptiveSparkPlanExec.scala:359)
	at org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanExec.withFinalPlanUpdate(AdaptiveSparkPlanExec.scala:366)
	at org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanExec.doExecuteBroadcast(AdaptiveSparkPlanExec.scala:358)
	at org.apache.spark.sql.execution.SparkPlan.$anonfun$executeBroadcast$1(SparkPlan.scala:197)
	at org.apache.spark.sql.execution.SparkPlan.$anonfun$executeQuery$1(SparkPlan.scala:222)
	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
	at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:219)
	at org.apache.spark.sql.execution.SparkPlan.executeBroadcast(SparkPlan.scala:193)
	at org.apache.spark.sql.execution.SubqueryBroadcastExec.$anonfun$relationFuture$2(SubqueryBroadcastExec.scala:81)
	at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withExecutionId$1(SQLExecution.scala:139)
	at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:163)
	at org.apache.spark.sql.execution.SQLExecution$.withExecutionId(SQLExecution.scala:137)
	at org.apache.spark.sql.execution.SubqueryBroadcastExec.$anonfun$relationFuture$1(SubqueryBroadcastExec.scala:78)
	at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
	at scala.util.Success.$anonfun$map$1(Try.scala:255)
	at scala.util.Success.map(Try.scala:213)
	at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
	at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)

Steps/Code to reproduce bug
Create a partitioned table using Iceberg parquet data and run NDS query 7 against it using the Spark RAPIDS plugin (latest 22.06 version).

Expected behavior
Expectation is that query will succeed and not fail with an exception.

Environment details (please complete the following information)
Environment is local Linux desktop with RTX A5000 GPU. Spark version is 3.2.1 with standard benchmark configuration.

Additional context
N/A

@mattahrens mattahrens added bug Something isn't working ? - Needs Triage Need team to review and classify labels May 11, 2022
@mattahrens mattahrens changed the title [BUG] Query failure with java.lang.AssertionError inside org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanExec.doExecuteBroadcast when using partitioned Iceberg tables [BUG] Query failure with java.lang.AssertionError when using partitioned Iceberg tables May 11, 2022
@jlowe jlowe self-assigned this May 11, 2022
@sameerz sameerz removed the ? - Needs Triage Need team to review and classify label May 12, 2022
@sameerz sameerz added this to the May 2 - May 20 milestone May 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants