Skip to content

Commit

Permalink
make up some comments
Browse files Browse the repository at this point in the history
Signed-off-by: sperlingxx <lovedreamf@gmail.com>
  • Loading branch information
sperlingxx committed Aug 3, 2021
1 parent 66084d4 commit 28d3fda
Show file tree
Hide file tree
Showing 2 changed files with 10 additions and 0 deletions.
3 changes: 3 additions & 0 deletions integration_tests/src/main/python/hash_aggregate_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -399,6 +399,9 @@ def test_hash_reduction_pivot_without_nans(data_gen, conf):
('c', LongRangeGen())] for value_gen in _repeat_agg_column_for_collect_op
]

# We wrapped sort_array functions on collect_list/collect_set because the orders of collected lists/sets are not
# deterministic. The annotation `ignore_order` only affects on the order between rows, while with collect ops we also
# need to guarantee the consistency of the row-wise order (the orders within each array produced by collect ops).
@approximate_float
@ignore_order(local=True)
@incompat
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -981,6 +981,13 @@ abstract class GpuTypedImperativeSupportedAggregateExecMeta[INPUT <: SparkPlan](
// are inconsistent with CPU buffers. Therefore, we have to fall back all Aggregate stages
// to CPU once any of them did fallback, in order to guarantee no partial-accelerated
// TypedImperativeAggregate function.
//
// This fallback procedure adapts AQE. As what GpuExchanges do, it leverages the
// `gpuSupportedTag` to store the information about whether instances are GPU-supported
// or not, which is produced by the side effect of `willNotWorkOnGpu`. When AQE is on,
// during the preparation stage, there will be a run of GpuOverrides on the entire plan to
// trigger these side effects if necessary, before AQE splits the entire query into several
// query stages.
GpuTypedImperativeSupportedAggregateExecMeta.checkAndFallbackEntirely(this)
}

Expand Down

0 comments on commit 28d3fda

Please sign in to comment.