[BUG] disable collect_list and collect_set for window operations by default. #10110
Labels
bug
Something isn't working
reliability
Features to improve reliability or bugs that severly impact the reliability of the plugin
Describe the bug
collect_list
andcollect_set
in a window context are going to make memory explode (the size of the window squared).collect_set
right now is implemented in terms ofcollect_list
and a dedupe stage. So with how we are currently doing window operations there is no good way to try and makecollect_list
orcollect_set
work in the context of a window. We should disable them by default until we have a better solution.The text was updated successfully, but these errors were encountered: