Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Compile error for Spark330 because of VectorizedColumnReader constructor added a new parameter. #5123

Closed
res-life opened this issue Apr 1, 2022 · 1 comment · Fixed by #5124
Assignees
Labels
audit_3.3.0 Audit related tasks for 3.3.0 bug Something isn't working

Comments

@res-life
Copy link
Collaborator

res-life commented Apr 1, 2022

Describe the bug

mvn clean install -Dbuildver=330 -DskipTests
not enough arguments for constructor VectorizedColumnReader

[INFO] Compiler bridge file: /home/chong/.sbt/1.0/zinc/org.scala-sbt/org.scala-sbt-compiler-bridge_2.12-1.3.1-bin_2.12.15__52.0-1.3.1_20191012T045515.jar
[INFO] Compiling 267 Scala sources and 29 Java sources to /home/chong/code/2-spark-rapids/sql-plugin/target/spark330/classes ...
[ERROR] [Error] /home/chong/code/2-spark-rapids/sql-plugin/src/main/321+/scala/org/apache/spark/sql/execution/datasources/parquet/rapids/shims/ShimVectorizedColumnReader.scala:58: not enough arguments for constructor VectorizedColumnReader: (x$1: org.apache.parquet.column.ColumnDescriptor, x$2: org.apache.parquet.schema.LogicalTypeAnnotation, x$3: org.apache.parquet.column.page.PageReader, x$4: java.util.PrimitiveIterator.OfLong, x$5: java.time.ZoneId, x$6: String, x$7: String, x$8: String, x$9: String, x$10: org.apache.parquet.VersionParser.ParsedVersion)org.apache.spark.sql.execution.datasources.parquet.VectorizedColumnReader.
Unspecified value parameter x$10.
[ERROR] [Error] /home/chong/code/2-spark-rapids/sql-plugin/src/main/321+/scala/org/apache/spark/sql/execution/datasources/parquet/rapids/shims/ShimVectorizedColumnReader.scala:20: Unused import
[ERROR] two errors found
@res-life res-life added bug Something isn't working ? - Needs Triage Need team to review and classify audit_3.3.0 Audit related tasks for 3.3.0 labels Apr 1, 2022
@pxLi
Copy link
Collaborator

pxLi commented Apr 1, 2022

related to apache/spark#35262 which affected 330+

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
audit_3.3.0 Audit related tasks for 3.3.0 bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants