[REVIEW] Add support for LEAD/LAG window functions for fixed-width types [redux] #6277
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Please update the changelog in order to start CI tests. View the gpuCI docs here. |
mythrocks
force-pushed
the
lead-lag-branch-0.16-redux
branch
from
06af3db
to
2caa5e6
Compare
mythrocks
requested review from
hyperbolic2346
and removed request for
rgsl888prabhu
Initial implementation for LEAD/LAG window functions, for fixed-width types. This version is implemented within grouped_rolling_window().
mythrocks
force-pushed
the
lead-lag-branch-0.16-redux
branch
from
2caa5e6
to
a56a106
Compare
Codecov Report
@@ Coverage Diff @@
## branch-0.16 #6277 +/- ##
============================================
Coverage 83.00% 83.00%
============================================
Files 95 95
Lines 14901 14901
============================================
Hits 12368 12368
Misses 2533 2533 Continue to review full report at Codecov.
|
mythrocks
changed the title
harrism
changed the title
|
@jrhemstad, does this PR look like a more acceptable approach than #6261? Also, @rgsl888prabhu, could I please bother you to have a look? |
1. Fixed includes 2. Un-templatized DeviceLeadLag 3. Documentation fixes
rgsl888prabhu
requested changes
Sep 24, 2020
jrhemstad
reviewed
Sep 24, 2020
|
rerun tests |
rgsl888prabhu
approved these changes
Oct 1, 2020
jrhemstad
requested changes
Oct 5, 2020
1. Moved Lead/Lag specific error checks out of rolling_window() 2. Moved Lead/Lag specific short-circuit evaluation out of rolling_window() 3. Added tests for error-checking
|
@jrhemstad, have the concerns raised here been addressed satisfactorily? |
jrhemstad
reviewed
Oct 7, 2020
jrhemstad
reviewed
Oct 7, 2020
1. Remove redundant checks for LEAD/LAG in non-LEAD/LAG SFINAE path 2. Remove hardcode for stream id
jrhemstad
approved these changes
Oct 7, 2020
|
Thank you for the review, and the great catches, @jrhemstad. |
rapids-bot bot
pushed a commit
that referenced
this pull request
May 3, 2021
This commit extends the [LEAD()/LAG() offset window functions](https://learnsql.com/blog/lead-and-lag-functions-in-sql/) introduced in #6277 to work with more than just fixed-width data types. The functions are defined as follows: 1. `LEAD(N)`: Returns the row from the input column, at the specified offset past the current row. If the offset crosses the grouping boundary or column boundary for a given row, a "default" value is returned if available. The "default" value is null, by default. 1. `LAG(N)`: returns the row from the input column at the specified offset preceding the current row. If the offset crosses the grouping boundary or column boundary for a given row, a "default" value is returned if available. The "default" value is null, by default. As an illustration, consider the following example input array input column, with two groups (`G1` and `G2`): ```c++ [ [1,1,1], [2], [3,3,3], [4,4,4,4], [66,66], [], [88,88], [99] ] <--------------G1--------------> <------------G2-----------> ``` `LEAD(col, 1)` yields: ```c++ [ [2], [3,3,3], [4,4,4,4], ∅, [], [88,88], [99], ∅ ] ``` `LAG(input_col, 2)` yields: ```c++ [ ∅, ∅, [2], [3,3,3], ∅, ∅, [], [88,88] ] ``` If a `defaults` column is specified with contents: ```c++ [ [999], [999], [999], [999], [999], [999], [999], [999] ] ``` then, `LEAD(col, 1, defaults)` yields: ```c++ [ [2], [3,3,3], [4,4,4,4], [999], [], [88,88], [99], [999] ] ``` Note that in the cases where the offset (`1`) would cross the column/group boundary (i.e. indices `3` and `7`), the corresponding entry from the `defaults` column is returned instead of `∅` (i.e. `null`). Authors: - MithunR (https://github.com/mythrocks) Approvers: - Nghia Truong (https://github.com/ttnghia) - Ram (Ramakrishna Prabhu) (https://github.com/rgsl888prabhu) - Karthikeyan (https://github.com/karthikeyann) URL: #8062
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is an alternative implementation for what #6261 attempts to solve, i.e.
LEAD()/LAG()offset window functions for fixed-width data types.Offset-based window functions include LEAD and LAG:
LEAD(N): Returns the row from the input column, at the specified offset past thecurrent row. If the offset crosses the grouping boundary or column boundary for
a given row, a "default" value is returned if available. The "default" value is
null, by default.
LAG(N): returns the row from the input column at the specified offset precedingthe current row. If the offset crosses the grouping boundary or column boundary for
a given row, a "default" value is returned if available. The "default" value is
null, by default.
E.g. Consider an input column with the following values and grouping:
[10, 11, 12, 13, 20, 21, 22, 23]<------G1-----> <------G2------>LEAD(input_col, 1)yields:[11, 12, 13, null, 21, 22, 23, null]LAG(input_col, 2)yields:[null, null, 11, 12, null, null, 21, 22]LEAD(input_col, 1, 99)(where 99 indicates the default) yields:[11, 12, 13, 99, 21, 22, 23, 99]