Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] databricks nightly window_function_test failures #4936

Closed
tgravescs opened this issue Mar 11, 2022 · 3 comments · Fixed by #4953
Closed

[BUG] databricks nightly window_function_test failures #4936

tgravescs opened this issue Mar 11, 2022 · 3 comments · Fixed by #4953
Assignees
Labels
bug Something isn't working P0 Must have for release

Comments

@tgravescs
Copy link
Collaborator

tgravescs commented Mar 11, 2022

Describe the bug

[2022-03-11T03:51:14.568Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_rank[Decimal(20,2)][IGNORE_ORDER({'local': True})]
[2022-03-11T03:51:14.568Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Decimal(20,2)-Long-1000][IGNORE_ORDER({'local': True})]
[2022-03-11T03:51:14.568Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Decimal(20,2)-Long-1g][IGNORE_ORDER({'local': True})]

Perhaps related to #4924?

[2022-03-11T03:51:14.566Z] java.lang.IllegalArgumentException: Part of the plan is not columnar class com.databricks.sql.execution.window.RunningWindowFunctionExec
[2022-03-11T03:51:14.566Z] RunningWindowFunction [b#215344, a#215343L, row_number() windowspecdefinition(b#215344, a#215343L ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS row_num#215352, rank(a#215343L) windowspecdefinition(b#215344, a#215343L ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS rank_val#215353, dense_rank(a#215343L) windowspecdefinition(b#215344, a#215343L ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS dense_rank_val#215354, count(c#215345L) windowspecdefinition(b#215344, a#215343L ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS count_col#215355L, min(c#215345L) windowspecdefinition(b#215344, a#215343L ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS min_col#215356L, max(c#215345L) windowspecdefinition(b#215344, a#215343L ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS max_col#215357L, sum(c#215345L) windowspecdefinition(b#215344, a#215343L ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS sum_col#215358L], [b#215344], [a#215343L ASC NULLS FIRST]
[2022-03-11T03:51:14.567Z] +- GpuColumnarToRow false
[2022-03-11T03:51:14.567Z]    +- GpuSort [b#215344 ASC NULLS FIRST, a#215343L ASC NULLS FIRST], false, com.nvidia.spark.rapids.OutOfCoreSort$@269e4d10
[2022-03-11T03:51:14.567Z]       +- GpuShuffleCoalesce 1073741824
[2022-03-11T03:51:14.567Z]          +- GpuColumnarExchange gpuhashpartitioning(b#215344, 12), true, [id=#264892]
[2022-03-11T03:51:14.567Z]             +- GpuProject [b#215344, a#215343L, c#215345L]
[2022-03-11T03:51:14.567Z]                +- GpuRowToColumnar targetsize(1073741824)
[2022-03-11T03:51:14.567Z]                   +- Scan ExistingRDD[a#215343L,b#215344,c#215345L]
@tgravescs tgravescs added bug Something isn't working ? - Needs Triage Need team to review and classify P0 Must have for release labels Mar 11, 2022
@NVnavkumar NVnavkumar self-assigned this Mar 11, 2022
@sameerz
Copy link
Collaborator

sameerz commented Mar 14, 2022

Since PR #4942 is a workaround, can we lower the priority of this issue?

Is there a long term fix we should track here?

@NVnavkumar
Copy link
Collaborator

I have a better fix than this workaround almost done which will track the actual xfail properly.

@tgravescs
Copy link
Collaborator Author

we should not lower the priority as in the very least we should xfail and ideally we fixed Databricks so it supports it properly or if we don't want to somehow indicate in our docs its not supported there.

@sameerz sameerz removed the ? - Needs Triage Need team to review and classify label Mar 15, 2022
@sameerz sameerz changed the title [BUG]databricks nightly window_function_test failures [BUG] databricks nightly window_function_test failures Apr 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working P0 Must have for release
Projects
None yet
3 participants