Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Watcher] Prevent expensive queries on ES by using PointInTimeFinder to get indexPatterns #119717

Merged
merged 12 commits into from
Dec 3, 2021
Merged

[Watcher] Prevent expensive queries on ES by using PointInTimeFinder to get indexPatterns #119717

merged 12 commits into from
Dec 3, 2021

Conversation

sabarasaba
Copy link
Member

@sabarasaba sabarasaba commented Nov 25, 2021 •
edited
Loading

Addresses #93770 in which Watcher could potentially create expensive queries to ES.

Summary

  • Size parameter was changed from 10000 to 1000
  • A new server side route that uses createPointInTimeFinder for paging. The client side code only uses the indexPattern names, so we can just return that from the server and avoid sending unnecessary information.

@sabarasaba sabarasaba self-assigned this Nov 25, 2021
@sabarasaba sabarasaba added backport:skip This commit does not require backporting chore Feature:Watcher Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more v8.1.0 release_note:skip Skip the PR/issue when compiling release notes labels Nov 29, 2021
@sabarasaba
Copy link
Member Author

@elasticmachine merge upstream

@sabarasaba sabarasaba marked this pull request as ready for review November 30, 2021 10:00
@sabarasaba sabarasaba requested a review from a team as a code owner November 30, 2021 10:00
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-stack-management (Team:Stack Management)

@sabarasaba
Copy link
Member Author

@elasticmachine merge upstream

alexwizp
alexwizp approved these changes Dec 1, 2021
View reviewed changes
Copy link
Contributor

@alexwizp alexwizp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code LGTM! Thank you

@sebelga sebelga self-requested a review December 2, 2021 09:14
sebelga
sebelga approved these changes Dec 2, 2021
View reviewed changes
Copy link
Contributor

@sebelga sebelga left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for fixing this @sabarasaba. Tested locally and works as expected 👍 Left a tiny comment about at type.

x-pack/plugins/watcher/server/routes/api/indices/register_get_index_patterns_route.ts Outdated Show resolved Hide resolved
@sabarasaba
Copy link
Member Author

@elasticmachine merge upstream

@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
watcher 272.6KB 272.6KB -68.0B

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @sabarasaba

@sabarasaba sabarasaba added auto-backport Deprecated - use backport:version if exact versions are needed and removed backport:skip This commit does not require backporting labels Dec 3, 2021
@sabarasaba sabarasaba merged commit fd741b4 into elastic:main Dec 3, 2021
@kibanamachine kibanamachine added the backport:skip This commit does not require backporting label Dec 3, 2021
TinLe pushed a commit to TinLe/kibana that referenced this pull request Dec 22, 2021
@sabarasaba @kibanamachine @TinLe
[Watcher] Prevent expensive queries on ES by using PointInTimeFinder …
8038d91 
…to get indexPatterns (elastic#119717)

* Use PointInTimeFinder to paginate through indexPatterns in a more performant way

* commit using @elastic.co

* Start working on tests

* Add tests

* Add missing types

* Fix tests

* Update tests responses

* Remove unnecessary mocks

* Fix type

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
gbamparop pushed a commit to gbamparop/kibana that referenced this pull request Jan 12, 2022
@sabarasaba @kibanamachine @gbamparop
[Watcher] Prevent expensive queries on ES by using PointInTimeFinder …
2f0cacc 
…to get indexPatterns (elastic#119717)

* Use PointInTimeFinder to paginate through indexPatterns in a more performant way

* commit using @elastic.co

* Start working on tests

* Add tests

* Add missing types

* Fix tests

* Update tests responses

* Remove unnecessary mocks

* Fix type

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
@tylersmalley tylersmalley added ci:cloud-deploy Create or update a Cloud deployment and removed ci:deploy-cloud labels Aug 17, 2022
wayneseymour added a commit to wayneseymour/kibana that referenced this pull request Oct 11, 2024
@wayneseymour
[FTR][Ownership] Assign ally/grps 1,2 & 3
131d429 
Contributes to: elastic#194817

Why I assigned them:

Assigned advanced_settings due to elastic#175255

Assigned dashboard_controls due to elastic#190797

Assigned dashboard_links due to elastic#174772

Assigned dashboard_panel_options due to elastic#178596

Assigned grok_debugger due to https://github.com/elastic/kibana/blob/main/x-pack/plugins/grokdebugger/kibana.jsonc#L4

Assigned helpers due to elastic#164341 (call site)

Assigned home due to elastic#103192

Assigned index_lifecycle_management due to elastic#116207

Assigned ingest_node_pipelines due to elastic#113783

Assigned kibana_overview due to  https://github.com/elastic/kibana/blob/f00ac7a8a21463e6bb4a2784c3a3884f36c62900/x-pack/plugins/grokdebugger/kibana.jsonc#L4

Assigned management due to elastic#165705

Assigned painless_lab due to https://github.com/elastic/kibana/blob/main/x-pack/plugins/painless_lab/kibana.jsonc#L4

Assigned search_profiler due to elastic#195343

Assigned uptime due to https://github.com/elastic/kibana/blob/main/x-pack/plugins/observability_solution/uptime/kibana.jsonc#L4

Assigned lens due to elastic#175893

Assigned ml_anomaly_detection due to elastic#162126

Assigned canvas due to elastic#164376

Assigned cc replication due to elastic#149069

Assigned enterpise search due to elastic#79359

Assigned graph to due elastic#190797

Assigned license_management due to https://github.com/elastic/kibana/blob/main/x-pack/plugins/license_management/kibana.jsonc#L4

Assigned maps due to elastic#155161

Assigned observability due to https://github.com/elastic/kibana/blob/main/x-pack/plugins/observability_solution/observability/kibana.jsonc#L4

Assigned remote clusters due to elastic#96989

Assigned reporting due to elastic#121435

Assigned rollup_jobs due to https://github.com/elastic/kibana/blob/d57bc9a5d7d64f86b550eff7997605a3090aee9a/x-pack/plugins/rollup/kibana.jsonc#L4

Assigned watcher due to elastic#119717
@wayneseymour wayneseymour mentioned this pull request Oct 11, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sebelga sebelga sebelga approved these changes

@alexwizp alexwizp alexwizp approved these changes

Assignees

@sabarasaba sabarasaba

Labels
auto-backport Deprecated - use backport:version if exact versions are needed backport:skip This commit does not require backporting chore ci:cloud-deploy Create or update a Cloud deployment Feature:Watcher release_note:skip Skip the PR/issue when compiling release notes Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more v8.1.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@sabarasaba @elasticmachine @kibana-ci @sebelga @alexwizp @tylersmalley @kibanamachine