-
Notifications
You must be signed in to change notification settings - Fork 24.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Performance degradation for '*' query with '_all' field disabled #25556
Comments
Relates to elastic/kibana#12097, Kibana needs to change to using |
It's true that kibana should default to the match_all query on discovery only limited by the time range, but on the other hand, in this case, a '*' query is not going to mean a |
I agree, I believe that we should special case {
"query": {
"query_string": {
"query": "*"
}
}
} |
I think we should do #25551 first and then we can have a special case for:
... which seems more natural since it will rewrite this query as a real |
+1. A interface-layer version of this saves me 20s on 89 million records. Thanks for bringing up the issue, I wasn't sure if the performance regression I was seeing was limited to '*' |
👍 This also has one additional side effect on Kibana/Grafana if you have a lot of visualizations/dashboards already created they have been stored in ES with the
which will cause all your visualizations/dashboards to become slower. |
This commit changes the way we handle field expansion in `match`, `multi_match` and `query_string` query. The main changes are: - For exact field name, the new behavior is to rewrite to a matchnodocs query when the field name is not found in the mapping. - For partial field names (with `*` suffix), the expansion is done only on `keyword`, `text`, `date` and `number` field types. Other field types are simply ignored. - For all fields (`*`), the expansion is done on accepted field types only (see above) and metadata fields are also filtered. - The `*` notation can also be used to set `default_field` option on`query_string` query. This should replace the needs for the extra option `use_all_fields` which is deprecated in this change. This commit also rewrites simple `*` query to matchalldocs query when all fields are requested (Fixes elastic#25556). The same change should be done on `simple_query_string` for completeness. Relates elastic#25551
Related to elastic#25726, this resolves elastic#25556 for the 5.x series by parsing "*" as a `MatchAllDocsQuery` instead of expanding it to a (potentially expensive) query on the `_field_names`.
Elasticsearch version: 5.4.0 - 5.4.3
Plugins installed: []
JVM version:
java version "1.8.0_60"
Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
OS version:
Linux debian-elasticseearch-client-test 3.16.0-4-amd64 #1 SMP Debian 3.16.39-1 (2016-12-30) x86_64 GNU/Linux
After upgrading to ES 5.4.3 the following query:
becomes really slow, after a little of investigation sending this query into to the validate API:
returns the following explanation:
Which means that is using the
_field_names
field causing the query to be extremely slow. By comparison the same query with the same index/mapping on ES 5.2.2-5.3.3 would output the following explanation:We've disabled the
_all
field. Disabling the_field_names
field doesn't help either, and we get this output on the validate API:This is especially problematic because the default query for discovery on Kibana is
*
, which is causing really slow response times in our setup. Looks like in previous versions, this query would've been rewritten to a MatchAllDocsQuery.The text was updated successfully, but these errors were encountered: