Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GroupBy V2 query fails when using a large limit #4739

Closed
jon-wei opened this issue Aug 31, 2017 · 1 comment
Closed

GroupBy V2 query fails when using a large limit #4739

jon-wei opened this issue Aug 31, 2017 · 1 comment
Assignees
Labels
Milestone

Comments

@jon-wei
Copy link
Contributor

jon-wei commented Aug 31, 2017

0.10.1 added limit push down optimization to GroupBy V2 in PR #3873.

When the optimization is on, GroupBy V2 requires a buffer with a minimum size depending on the limit.

If the user submits a query with a limit that's large enough such that the configured processing buffers are too small to support this optimization, the query fails with an error like:

 { "errorClass": "io.druid.java.util.common.IAE", "host": "localhost:8083", "errorMessage": "Buffer capacity [34636833] is too small for limit[500000000] with load factor[0.700000], minimum bytes needed: [1666887684]", "error": "Unknown exception" }

It would be better to disable the push down optimization in that situation instead of failing the query.

@jon-wei jon-wei added the Bug label Aug 31, 2017
@jon-wei jon-wei added this to the 0.11.0 milestone Aug 31, 2017
@jon-wei jon-wei self-assigned this Aug 31, 2017
@jon-wei
Copy link
Contributor Author

jon-wei commented Aug 31, 2017

As a workaround, the user can disable limit push down in the query context, e.g.:

"context": {"applyLimitPushDown": false }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant