Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove the unnecessary parameter dataType in resolveColumnVector method #4340

Merged
merged 1 commit into from
Dec 10, 2021
Merged

Remove the unnecessary parameter dataType in resolveColumnVector method #4340

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuExpandExec.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,7 @@ class GpuExpandIterator(
val (cv, nullColumnReused) = expr.columnarEval(cb) match {
case null => getOrCreateNullCV(sparkType)
case other =>
(GpuExpressionsUtils.resolveColumnVector(other, cb.numRows, sparkType), false)
(GpuExpressionsUtils.resolveColumnVector(other, cb.numRows), false)
GaryShen2008 marked this conversation as resolved.
Show resolved Hide resolved
}
if (!nullColumnReused) {
uniqueDeviceColumns += cv
Expand Down
11 changes: 5 additions & 6 deletions sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuExpressions.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ import com.nvidia.spark.rapids.shims.v2.{ShimBinaryExpression, ShimExpression, S
import org.apache.spark.sql.catalyst.InternalRow
import org.apache.spark.sql.catalyst.expressions._
import org.apache.spark.sql.catalyst.expressions.codegen.{CodegenContext, ExprCode}
import org.apache.spark.sql.types.{DataType, StringType}
import org.apache.spark.sql.types.StringType
import org.apache.spark.sql.vectorized.ColumnarBatch
import org.apache.spark.unsafe.types.UTF8String

Expand Down Expand Up @@ -64,15 +64,14 @@ object GpuExpressionsUtils extends Arm {
* should cover all the cases for GPU pipelines.
*
* @param any the input value. It will be closed if it is a closeable after the call done.
* @param numRows the expected row number of the output column, used when 'any' is a Scalar.
* @param dType the data type of the output column, used when 'any' is a Scalar.
* @param numRows the expected row number of the output column, used when 'any' is a GpuScalar.
* @return a `GpuColumnVector` if it succeeds. Users should close the column vector to avoid
* memory leak.
*/
def resolveColumnVector(any: Any, numRows: Int, dType: DataType): GpuColumnVector = {
def resolveColumnVector(any: Any, numRows: Int): GpuColumnVector = {
withResourceIfAllowed(any) {
case c: GpuColumnVector => c.incRefCount()
case s: GpuScalar => GpuColumnVector.from(s, numRows, dType)
case s: GpuScalar => GpuColumnVector.from(s, numRows, s.dataType)
case other =>
throw new IllegalArgumentException(s"Cannot resolve a ColumnVector from the value:" +
s" $other. Please convert it to a GpuScalar or a GpuColumnVector before returning.")
Expand All @@ -91,7 +90,7 @@ object GpuExpressionsUtils extends Arm {
* memory leak.
*/
def columnarEvalToColumn(expr: Expression, batch: ColumnarBatch): GpuColumnVector =
resolveColumnVector(expr.columnarEval(batch), batch.numRows, expr.dataType)
resolveColumnVector(expr.columnarEval(batch), batch.numRows)

/**
* Extract the GpuLiteral
Expand Down