Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DecimalType support for Aggregate Count #1476

Merged
merged 2 commits into from
Jan 8, 2021

Conversation

razajafri
Copy link
Collaborator

@razajafri razajafri commented Jan 8, 2021

Turned on the Decimal Type for Count on and added test

Signed-off-by: Raza Jafri rjafri@nvidia.com

This fixes #1334

Turned it on and added test

Signed-off-by: Raza Jafri <rjafri@nvidia.com>
jlowe
jlowe previously approved these changes Jan 8, 2021
Copy link
Member

@jlowe jlowe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. I was expecting a change to an existing test rather than a new test, but I couldn't find a test that just did a count without combining it with other aggregates and/or filters.

@jlowe jlowe added this to the Jan 4 - Jan 15 milestone Jan 8, 2021
@revans2
Copy link
Collaborator

revans2 commented Jan 8, 2021

build

@@ -418,3 +418,10 @@ def test_arithmetic_reductions(data_gen):
'avg(a)'),
conf = _no_nans_float_conf)

@ignore_order
@pytest.mark.parametrize('data_gen', all_gen, ids=idfn)
@pytest.mark.parametrize('count_func', [f.count, f.countDistinct], ids=idfn)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using a function in here with idfn is producing different names each time the test runs and is causing the parallel runs of the tests to fail. If you drop the ids=idfn it fixes the problem

idfn causes a problem when passing a function as a parameter to the test

Signed-off-by: Raza Jafri <rjafri@nvidia.com>
@revans2
Copy link
Collaborator

revans2 commented Jan 8, 2021

build

@revans2 revans2 merged commit 2eaad03 into NVIDIA:branch-0.4 Jan 8, 2021
@razajafri razajafri deleted the agg_count branch January 8, 2021 20:04
@sameerz sameerz added the feature request New feature or request label Jan 20, 2021
nartal1 pushed a commit to nartal1/spark-rapids that referenced this pull request Jun 9, 2021
Signed-off-by: Raza Jafri <rjafri@nvidia.com>
nartal1 pushed a commit to nartal1/spark-rapids that referenced this pull request Jun 9, 2021
Signed-off-by: Raza Jafri <rjafri@nvidia.com>
tgravescs pushed a commit to tgravescs/spark-rapids that referenced this pull request Nov 30, 2023
…IDIA#1476)

Signed-off-by: spark-rapids automation <70000568+nvauto@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEA] Add support for count aggregate on decimal
4 participants