Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added in StringRPad and StringLPad #445

Merged
merged 4 commits into from
Jul 28, 2020
Merged

Conversation

revans2
Copy link
Collaborator

@revans2 revans2 commented Jul 28, 2020

Adds in support for rpad and lpad

This fixes #438

This depends on some changes to cudf that just were merged in, but because of how CI works, it may be a while before these will build. I'll try to speed up the process.

Signed-off-by: Robert (Bobby) Evans <bobby@apache.org>
@revans2 revans2 added the SQL part of the SQL/Dataframe plugin label Jul 28, 2020
@revans2 revans2 added this to the Jul 20 - Jul 31 milestone Jul 28, 2020
@revans2 revans2 self-assigned this Jul 28, 2020
@@ -160,15 +160,15 @@ def with_special_pattern(self, pattern, flags=0, charset=sre_yield.CHARSET, weig
length = int(len(strs))
except OverflowError:
length = _MAX_CHOICES
return self.with_special_case(lambda rand : strs[rand.randint(0, length)], weight=weight)
return self.with_special_case(lambda rand : strs[rand.randrange(0, length)], weight=weight)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found this during debugging. randrange is not inclusive on the high end, but randint is

}
}
override def convertToGpu(
str: Expression,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: indentation is off here.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IntelliJ is setting that to the indentation with the settings we have published for it. Don't know if it is a regression in IntelliJ or what, but I'll fix it for these

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw these were updated, but the style guide doesn't call for aligning arguments with the open parenthesis. Instead it should be a continuation (i.e.: 4-space indent from the start of the previous line). Apparently the style settings that are published are messed up, I'll look into it and fix if necessary.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked the IntelliJ settings file and it appears to be correct. Specifically it calls out these settings which should not have aligned the parameters on the opening parenthesis:

  <codeStyleSettings language="Scala">
    <option name="ALIGN_MULTILINE_PARAMETERS" value="false" />
    <indentOptions>
      <option name="CONTINUATION_INDENT_SIZE" value="4" />
    </indentOptions>
  </codeStyleSettings>

}
}
override def convertToGpu(
str: Expression,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Indentation is off here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This indentation is still off.

override def children: Seq[Expression] = str :: len :: pad :: Nil
override def dataType: DataType = StringType
override def inputTypes: Seq[DataType] = Seq(StringType, IntegerType, StringType)
override def prettyName: String = "lpad"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be overridden in the derived classes rather than here?

}

override def doColumnar(
str: GpuColumnVector,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: indentation is off here.

throw new IllegalStateException("This is not supported yet")

override def doColumnar(
str: Scalar,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: indentation

throw new IllegalStateException("This is not supported yet")

override def doColumnar(
str: GpuColumnVector,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: indentation

throw new IllegalStateException("This is not supported yet")

override def doColumnar(
str: GpuColumnVector,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: indentation

Signed-off-by: Robert (Bobby) Evans <bobby@apache.org>
@revans2
Copy link
Collaborator Author

revans2 commented Jul 28, 2020

build

Signed-off-by: Robert (Bobby) Evans <bobby@apache.org>
@revans2
Copy link
Collaborator Author

revans2 commented Jul 28, 2020

build

Signed-off-by: Robert (Bobby) Evans <bobby@apache.org>
@revans2
Copy link
Collaborator Author

revans2 commented Jul 28, 2020

build

@revans2 revans2 merged commit c95ad7c into NVIDIA:branch-0.2 Jul 28, 2020
nartal1 pushed a commit to nartal1/spark-rapids that referenced this pull request Jun 9, 2021
nartal1 pushed a commit to nartal1/spark-rapids that referenced this pull request Jun 9, 2021
pxLi pushed a commit to pxLi/spark-rapids that referenced this pull request May 12, 2022
tgravescs pushed a commit to tgravescs/spark-rapids that referenced this pull request Nov 30, 2023
Signed-off-by: spark-rapids automation <70000568+nvauto@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
SQL part of the SQL/Dataframe plugin
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEA] Enable StringLPad
2 participants