[Search service] Asynchronous ES search strategy #53538

lukasolson · 2019-12-18T22:28:08Z

Dev Docs

Adds an async ES search strategy which utilizes the async search strategy to call the _async_search APIs in Elasticsearch, returning an observable over the responses.

To do

Set defaults (on both client & server)
Use the wait_for_completion parameter to ensure the first request may return quickly
Add cancel method that sends DELETE request to ES
Unit tests
Add dev docs

elasticmachine · 2019-12-18T22:28:12Z

Pinging @elastic/kibana-app-arch (Team:AppArch)

lizozom · 2020-03-12T16:17:05Z

x-pack/plugins/data_enhanced/server/search/es_search_strategy.ts

+  const path = request.id ? `_async_search/${request.id}` : `${index}/_async_search`;
+
+  // Wait up to 1s for the response to return
+  const query = toSnakeCase({ ...params, waitForCompletion: '1s' });


Should the 1s interval be a setting?

Like a UI setting? It's something we can discuss. I've updated it to be configurable by the caller.

lizozom

Overall LGTM
Would rename ICancel + make the async wait time configurable

lizozom · 2020-03-17T10:47:25Z

@elasticmachine merge upstream

lizozom · 2020-03-17T19:44:18Z

@elasticmachine merge upstream

lukasolson · 2020-03-17T23:35:59Z

@elasticmachine merge upstream

* Add async search strategy * Add async search * Fix async strategy and add tests * Move types to separate file * Revert changes to demo search * Update demo search strategy to use async * Add async es search strategy * Return response as rawResponse * Poll after initial request * Add cancellation to search strategies * Add tests * Simplify async search strategy * Move loadingCount to search strategy * Update abort controller library * Bootstrap * Abort when the request is aborted * Add utility and update value suggestions route * Fix bad merge conflict * Update tests * Move to data_enhanced plugin * Remove bad merge * Revert switching abort controller libraries * Revert package.json in lib * Move to previous abort controller * Add support for frozen indices * Fix test to use fake timers to run debounced handlers * Revert changes to example plugin * Fix loading bar not going away when cancelling * Call getSearchStrategy instead of passing directly * Add async demo search strategy * Fix error with setting state * Update how aborting works * Fix type checks * Add test for loading count * Attempt to fix broken example test * Revert changes to test * Fix test * Update name to camelCase * Fix failing test * Don't require data_enhanced in example plugin * Actually send DELETE request * Use waitForCompletion parameter * Use default search params * Add support for rollups * Only make changes needed for frozen indices/rollups * Only make changes needed for frozen indices/rollups * Add back in async functionality * Fix tests/types * Fix issue with sending empty body in GET * Don't include skipped in loaded/total * Don't wait before polling the next time * Simplify search logic * Fix merge error * Review feedback * Fix issue with hits.total Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

* master: [ML] Re-enabling file upload telemetry (elastic#60418) [NP] Use local helper shortenDottedString for discover (elastic#60271) [Console] Fix for `_settings` and x-pack autocomplete (elastic#60246) Task/host enhancements (elastic#59671) [Search service] Asynchronous ES search strategy (elastic#53538) Index Action - Moved index params fields to connector config (elastic#60349) Edits UI text for ML nodes and job button (elastic#60184) Publish getIsNavDrawerLocked$ method on core chrome service. (elastic#60191) Disabled edit alert button on management ui for non registered UI alert types (elastic#60439) Revert "[Console] Fix bool filter autocompletions and refactor (elastic#60361)" [Console] Fix bool filter autocompletions and refactor (elastic#60361) Update ingest management team handle (elastic#60457) [IM] Use EuiCodeBlock to render index mapping (elastic#60420) Add additional safeguards for data source wizard step 2 (elastic#60426) [kbn/pm] don't fail when plugins are outside repo (elastic#60164) upgrade react-use (elastic#60427) Remove link to old settings (elastic#60326) Update app arch CODEOWNERS items. (elastic#60396) [ML] Fixing custom urls to dashboards (elastic#60355) Update the ems-client dependency to 7.7.0 (elastic#59936)

kibanamachine · 2020-03-19T01:48:44Z

Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync.

…alerting/tls-warning * 'alerting/tls-warning' of github.com:gmmorris/kibana: (33 commits) [ML] Disable functional transform tests Fixes to service map single node banner (elastic#60072) [Uptime] replace fetch with kibana http (elastic#59881) Upgrade @types/node to match Node.js runtime (elastic#60368) [License Management] NP migration (elastic#60250) Fix create alert button from not showing in alerts list (elastic#60444) [SIEM][Case] Update connector through flyout (elastic#60307) add data-test-subj where possible on SO management table (elastic#60226) Enforce `required` presence for value/key validation of `recordOf` and `mapOf`. (elastic#60406) [ML] Re-enabling file upload telemetry (elastic#60418) [NP] Use local helper shortenDottedString for discover (elastic#60271) [Console] Fix for `_settings` and x-pack autocomplete (elastic#60246) Task/host enhancements (elastic#59671) [Search service] Asynchronous ES search strategy (elastic#53538) Index Action - Moved index params fields to connector config (elastic#60349) Edits UI text for ML nodes and job button (elastic#60184) Publish getIsNavDrawerLocked$ method on core chrome service. (elastic#60191) Disabled edit alert button on management ui for non registered UI alert types (elastic#60439) Revert "[Console] Fix bool filter autocompletions and refactor (elastic#60361)" [Console] Fix bool filter autocompletions and refactor (elastic#60361) ...

kibanamachine · 2020-03-20T01:58:44Z

Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync.

* Add async search strategy * Add async search * Fix async strategy and add tests * Move types to separate file * Revert changes to demo search * Update demo search strategy to use async * Add async es search strategy * Return response as rawResponse * Poll after initial request * Add cancellation to search strategies * Add tests * Simplify async search strategy * Move loadingCount to search strategy * Update abort controller library * Bootstrap * Abort when the request is aborted * Add utility and update value suggestions route * Fix bad merge conflict * Update tests * Move to data_enhanced plugin * Remove bad merge * Revert switching abort controller libraries * Revert package.json in lib * Move to previous abort controller * Add support for frozen indices * Fix test to use fake timers to run debounced handlers * Revert changes to example plugin * Fix loading bar not going away when cancelling * Call getSearchStrategy instead of passing directly * Add async demo search strategy * Fix error with setting state * Update how aborting works * Fix type checks * Add test for loading count * Attempt to fix broken example test * Revert changes to test * Fix test * Update name to camelCase * Fix failing test * Don't require data_enhanced in example plugin * Actually send DELETE request * Use waitForCompletion parameter * Use default search params * Add support for rollups * Only make changes needed for frozen indices/rollups * Only make changes needed for frozen indices/rollups * Add back in async functionality * Fix tests/types * Fix issue with sending empty body in GET * Don't include skipped in loaded/total * Don't wait before polling the next time * Simplify search logic * Fix merge error * Review feedback * Fix issue with hits.total Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

kibanamachine · 2020-04-10T18:41:09Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request
Commit: 16bbeef

Test Failures

Kibana Pipeline / kibana-oss-agent / Chrome UI Functional Tests.test/functional/apps/discover/_errors·js.discover app errors invalid scripted field error is rendered

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has failed 1 times on tracked branches: https://dryrun

[00:00:00]       │
[00:01:37]         └-: discover app
[00:01:37]           └-> "before all" hook
[00:01:37]           └-> "before all" hook
[00:09:14]           └-: errors
[00:09:14]             └-> "before all" hook
[00:09:14]             └-> "before all" hook
[00:09:14]               │ info [logstash_functional] Loading "mappings.json"
[00:09:14]               │ info [logstash_functional] Loading "data.json.gz"
[00:09:14]               │ info [logstash_functional] Skipped restore for existing index "logstash-2015.09.22"
[00:09:14]               │ info [logstash_functional] Skipped restore for existing index "logstash-2015.09.20"
[00:09:14]               │ info [logstash_functional] Skipped restore for existing index "logstash-2015.09.21"
[00:09:14]               │ info [invalid_scripted_field] Loading "mappings.json"
[00:09:14]               │ info [invalid_scripted_field] Loading "data.json.gz"
[00:09:14]               │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] [.kibana_1/cle--3RAQKOUW3ze_HI4RQ] deleting index
[00:09:14]               │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] [.kibana_2/h2BhTR9ETR6CE4XL1avCbg] deleting index
[00:09:14]               │ info [invalid_scripted_field] Deleted existing index [".kibana_2",".kibana_1"]
[00:09:14]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] applying create index request using v1 templates []
[00:09:14]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] [.kibana] creating index, cause [api], templates [], shards [1]/[0], mappings [_doc]
[00:09:14]               │ info [invalid_scripted_field] Created index ".kibana"
[00:09:14]               │ debg [invalid_scripted_field] ".kibana" settings {"index":{"number_of_replicas":"0","number_of_shards":"1"}}
[00:09:14]               │ info [invalid_scripted_field] Indexed 2 docs into ".kibana"
[00:09:14]               │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] [.kibana/SqQkAgLhRlSRR_a2FmevNw] update_mapping [_doc]
[00:09:14]               │ debg Migrating saved objects
[00:09:14]               │ proc [kibana]   log   [17:34:27.651] [info][savedobjects-service] Creating index .kibana_2.
[00:09:14]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] applying create index request using v1 templates []
[00:09:14]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] [.kibana_2] creating index, cause [api], templates [], shards [1]/[1], mappings [_doc]
[00:09:14]               │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] updating number_of_replicas to [0] for indices [.kibana_2]
[00:09:14]               │ proc [kibana]   log   [17:34:27.685] [info][savedobjects-service] Reindexing .kibana to .kibana_1
[00:09:14]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] applying create index request using v1 templates []
[00:09:14]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] [.kibana_1] creating index, cause [api], templates [], shards [1]/[1], mappings [_doc]
[00:09:14]               │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] updating number_of_replicas to [0] for indices [.kibana_1]
[00:09:15]               │ info [o.e.t.LoggingTaskListener] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] 8403 finished with response BulkByScrollResponse[took=13.5ms,timed_out=false,sliceId=null,updated=0,created=2,deleted=0,batches=1,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[00:09:15]               │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] [.kibana/SqQkAgLhRlSRR_a2FmevNw] deleting index
[00:09:15]               │ proc [kibana]   log   [17:34:27.993] [info][savedobjects-service] Migrating .kibana_1 saved objects to .kibana_2
[00:09:15]               │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] [.kibana_2/BBDrZo3bRGmR5UjOGESxxg] update_mapping [_doc]
[00:09:15]               │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] [.kibana_2/BBDrZo3bRGmR5UjOGESxxg] update_mapping [_doc]
[00:09:15]               │ proc [kibana]   log   [17:34:28.035] [info][savedobjects-service] Pointing alias .kibana to .kibana_2.
[00:09:15]               │ proc [kibana]   log   [17:34:28.056] [info][savedobjects-service] Finished in 406ms.
[00:09:15]               │ debg applying update to kibana config: {"accessibility:disableAnimations":true,"dateFormat:tz":"UTC"}
[00:09:15]               │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] [.kibana_2/BBDrZo3bRGmR5UjOGESxxg] update_mapping [_doc]
[00:09:16]               │ debg navigating to discover url: http://localhost:6161/app/kibana#/discover
[00:09:16]               │ debg Navigate to: http://localhost:6161/app/kibana#/discover
[00:09:17]               │ debg ... sleep(700) start
[00:09:17]               │ debg browser[INFO] http://localhost:6161/app/kibana?_t=1586540069693#/discover 340 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:09:17]               │
[00:09:17]               │ debg browser[INFO] http://localhost:6161/bundles/app/kibana/bootstrap.js 9:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:09:17]               │ debg ... sleep(700) end
[00:09:17]               │ debg returned from get, calling refresh
[00:09:17]               │ debg browser[INFO] http://localhost:6161/app/kibana?_t=1586540069693#/discover 340 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:09:17]               │
[00:09:17]               │ debg browser[INFO] http://localhost:6161/bundles/app/kibana/bootstrap.js 9:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:09:18]               │ debg currentUrl = http://localhost:6161/app/kibana#/discover
[00:09:18]               │          appUrl = http://localhost:6161/app/kibana#/discover
[00:09:18]               │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=60000
[00:09:20]               │ debg TestSubjects.find(kibanaChrome)
[00:09:20]               │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=10000
[00:09:20]               │ debg browser[INFO] http://localhost:6161/bundles/plugin/data/data.plugin.js 96:139970 "INFO: 2020-04-10T17:34:32Z
[00:09:20]               │        Adding connection to http://localhost:6161/elasticsearch
[00:09:20]               │
[00:09:20]               │      "
[00:09:20]               │ debg ... sleep(501) start
[00:09:20]               │ debg ... sleep(501) end
[00:09:20]               │ debg in navigateTo url = http://localhost:6161/app/kibana#/discover?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-15m,to:now))&_a=(columns:!(_source),filters:!(),index:%2718ea0c30-2d77-11e8-93f2-6f8a391f33ef%27,interval:auto,query:(language:kuery,query:%27%27),sort:!())
[00:09:20]               │ debg --- retry.try error: URL changed, waiting for it to settle
[00:09:21]               │ debg ... sleep(501) start
[00:09:21]               │ debg ... sleep(501) end
[00:09:21]               │ debg in navigateTo url = http://localhost:6161/app/kibana#/discover?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-15m,to:now))&_a=(columns:!(_source),filters:!(),index:%2718ea0c30-2d77-11e8-93f2-6f8a391f33ef%27,interval:auto,query:(language:kuery,query:%27%27),sort:!())
[00:09:21]               │ debg TestSubjects.exists(statusPageContainer)
[00:09:21]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="statusPageContainer"]') with timeout=2500
[00:09:24]               │ debg --- retry.tryForTime error: [data-test-subj="statusPageContainer"] is not displayed
[00:09:24]             └-: invalid scripted field error
[00:09:24]               └-> "before all" hook
[00:09:24]               └-> is rendered
[00:09:24]                 └-> "before each" hook: global before each
[00:09:24]                 │ debg TestSubjects.exists(discoverFetchError)
[00:09:24]                 │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="discoverFetchError"]') with timeout=2500
[00:09:27]                 │ debg --- retry.tryForTime error: [data-test-subj="discoverFetchError"] is not displayed
[00:09:28]                 │ info Taking screenshot "/dev/shm/workspace/kibana/test/functional/screenshots/failure/discover app errors invalid scripted field error is rendered.png"
[00:09:28]                 │ info Current URL is: http://localhost:6161/app/kibana#/discover?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-15m,to:now))&_a=(columns:!(_source),filters:!(),index:%2718ea0c30-2d77-11e8-93f2-6f8a391f33ef%27,interval:auto,query:(language:kuery,query:%27%27),sort:!())
[00:09:28]                 │ info Saving page source to: /dev/shm/workspace/kibana/test/functional/failure_debug/html/discover app errors invalid scripted field error is rendered.html
[00:09:28]                 └- ✖ fail: "discover app errors invalid scripted field error is rendered"
[00:09:28]                 │

Stack Trace

Error: expected false to equal true
    at Assertion.assert (packages/kbn-expect/expect.js:100:11)
    at Assertion.be.Assertion.equal (packages/kbn-expect/expect.js:227:8)
    at Assertion.be (packages/kbn-expect/expect.js:69:22)
    at Context.it (test/functional/apps/discover/_errors.js:41:40)

Kibana Pipeline / kibana-oss-agent / Chrome UI Functional Tests.test/functional/apps/discover/_errors·js.discover app errors invalid scripted field error is rendered

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

[00:00:00]       │
[00:01:39]         └-: discover app
[00:01:39]           └-> "before all" hook
[00:01:40]           └-> "before all" hook
[00:09:40]           └-: errors
[00:09:40]             └-> "before all" hook
[00:09:40]             └-> "before all" hook
[00:09:40]               │ info [logstash_functional] Loading "mappings.json"
[00:09:40]               │ info [logstash_functional] Loading "data.json.gz"
[00:09:40]               │ info [logstash_functional] Skipped restore for existing index "logstash-2015.09.22"
[00:09:40]               │ info [logstash_functional] Skipped restore for existing index "logstash-2015.09.20"
[00:09:40]               │ info [logstash_functional] Skipped restore for existing index "logstash-2015.09.21"
[00:09:41]               │ info [invalid_scripted_field] Loading "mappings.json"
[00:09:41]               │ info [invalid_scripted_field] Loading "data.json.gz"
[00:09:41]               │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] [.kibana_1/FfiWKlFoSJ6_Hvbqm2vnaQ] deleting index
[00:09:41]               │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] [.kibana_2/C9fHirghT3SP96BrUHB_Fw] deleting index
[00:09:41]               │ info [invalid_scripted_field] Deleted existing index [".kibana_2",".kibana_1"]
[00:09:41]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] applying create index request using v1 templates []
[00:09:41]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] [.kibana] creating index, cause [api], templates [], shards [1]/[0], mappings [_doc]
[00:09:42]               │ info [invalid_scripted_field] Created index ".kibana"
[00:09:42]               │ debg [invalid_scripted_field] ".kibana" settings {"index":{"number_of_replicas":"0","number_of_shards":"1"}}
[00:09:42]               │ info [invalid_scripted_field] Indexed 2 docs into ".kibana"
[00:09:42]               │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] [.kibana/4sgjMMkdQL-OZuP4YiizbA] update_mapping [_doc]
[00:09:42]               │ debg Migrating saved objects
[00:09:42]               │ proc [kibana]   log   [17:23:20.297] [info][savedobjects-service] Creating index .kibana_2.
[00:09:42]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] applying create index request using v1 templates []
[00:09:42]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] [.kibana_2] creating index, cause [api], templates [], shards [1]/[1], mappings [_doc]
[00:09:42]               │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] updating number_of_replicas to [0] for indices [.kibana_2]
[00:09:42]               │ proc [kibana]   log   [17:23:20.350] [info][savedobjects-service] Reindexing .kibana to .kibana_1
[00:09:42]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] applying create index request using v1 templates []
[00:09:42]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] [.kibana_1] creating index, cause [api], templates [], shards [1]/[1], mappings [_doc]
[00:09:42]               │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] updating number_of_replicas to [0] for indices [.kibana_1]
[00:09:42]               │ info [o.e.t.LoggingTaskListener] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] 8460 finished with response BulkByScrollResponse[took=21.2ms,timed_out=false,sliceId=null,updated=0,created=2,deleted=0,batches=1,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[00:09:42]               │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] [.kibana/4sgjMMkdQL-OZuP4YiizbA] deleting index
[00:09:42]               │ proc [kibana]   log   [17:23:20.692] [info][savedobjects-service] Migrating .kibana_1 saved objects to .kibana_2
[00:09:42]               │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] [.kibana_2/Zg6f-prATsKvuaHuv6EByw] update_mapping [_doc]
[00:09:42]               │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] [.kibana_2/Zg6f-prATsKvuaHuv6EByw] update_mapping [_doc]
[00:09:42]               │ proc [kibana]   log   [17:23:20.747] [info][savedobjects-service] Pointing alias .kibana to .kibana_2.
[00:09:42]               │ proc [kibana]   log   [17:23:20.785] [info][savedobjects-service] Finished in 489ms.
[00:09:42]               │ debg applying update to kibana config: {"accessibility:disableAnimations":true,"dateFormat:tz":"UTC"}
[00:09:43]               │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-debian-tests-xl-1586537550550789486] [.kibana_2/Zg6f-prATsKvuaHuv6EByw] update_mapping [_doc]
[00:09:44]               │ debg navigating to discover url: http://localhost:6161/app/kibana#/discover
[00:09:44]               │ debg Navigate to: http://localhost:6161/app/kibana#/discover
[00:09:44]               │ debg ... sleep(700) start
[00:09:44]               │ debg browser[INFO] http://localhost:6161/app/kibana?_t=1586539402366#/discover 340 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:09:44]               │
[00:09:44]               │ debg browser[INFO] http://localhost:6161/bundles/app/kibana/bootstrap.js 9:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:09:44]               │ debg ... sleep(700) end
[00:09:44]               │ debg returned from get, calling refresh
[00:09:45]               │ debg browser[INFO] http://localhost:6161/app/kibana?_t=1586539402366#/discover 340 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:09:45]               │
[00:09:45]               │ debg browser[INFO] http://localhost:6161/bundles/app/kibana/bootstrap.js 9:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:09:45]               │ debg currentUrl = http://localhost:6161/app/kibana#/discover
[00:09:45]               │          appUrl = http://localhost:6161/app/kibana#/discover
[00:09:45]               │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=60000
[00:09:48]               │ debg TestSubjects.find(kibanaChrome)
[00:09:48]               │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=10000
[00:09:48]               │ debg browser[INFO] http://localhost:6161/bundles/plugin/data/data.plugin.js 96:139970 "INFO: 2020-04-10T17:23:25Z
[00:09:48]               │        Adding connection to http://localhost:6161/elasticsearch
[00:09:48]               │
[00:09:48]               │      "
[00:09:48]               │ debg ... sleep(501) start
[00:09:48]               │ debg ... sleep(501) end
[00:09:48]               │ debg in navigateTo url = http://localhost:6161/app/kibana#/discover?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-15m,to:now))&_a=(columns:!(_source),filters:!(),index:%2718ea0c30-2d77-11e8-93f2-6f8a391f33ef%27,interval:auto,query:(language:kuery,query:%27%27),sort:!())
[00:09:48]               │ debg --- retry.try error: URL changed, waiting for it to settle
[00:09:49]               │ debg ... sleep(501) start
[00:09:49]               │ debg ... sleep(501) end
[00:09:49]               │ debg in navigateTo url = http://localhost:6161/app/kibana#/discover?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-15m,to:now))&_a=(columns:!(_source),filters:!(),index:%2718ea0c30-2d77-11e8-93f2-6f8a391f33ef%27,interval:auto,query:(language:kuery,query:%27%27),sort:!())
[00:09:49]               │ debg TestSubjects.exists(statusPageContainer)
[00:09:49]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="statusPageContainer"]') with timeout=2500
[00:09:52]               │ debg --- retry.tryForTime error: [data-test-subj="statusPageContainer"] is not displayed
[00:09:53]             └-: invalid scripted field error
[00:09:53]               └-> "before all" hook
[00:09:53]               └-> is rendered
[00:09:53]                 └-> "before each" hook: global before each
[00:09:53]                 │ debg TestSubjects.exists(discoverFetchError)
[00:09:53]                 │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="discoverFetchError"]') with timeout=2500
[00:09:55]                 │ debg --- retry.tryForTime error: [data-test-subj="discoverFetchError"] is not displayed
[00:09:56]                 │ info Taking screenshot "/dev/shm/workspace/kibana/test/functional/screenshots/failure/discover app errors invalid scripted field error is rendered.png"
[00:09:56]                 │ info Current URL is: http://localhost:6161/app/kibana#/discover?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-15m,to:now))&_a=(columns:!(_source),filters:!(),index:%2718ea0c30-2d77-11e8-93f2-6f8a391f33ef%27,interval:auto,query:(language:kuery,query:%27%27),sort:!())
[00:09:56]                 │ info Saving page source to: /dev/shm/workspace/kibana/test/functional/failure_debug/html/discover app errors invalid scripted field error is rendered.html
[00:09:56]                 └- ✖ fail: "discover app errors invalid scripted field error is rendered"
[00:09:56]                 │

Stack Trace

Error: expected false to equal true
    at Assertion.assert (packages/kbn-expect/expect.js:100:11)
    at Assertion.be.Assertion.equal (packages/kbn-expect/expect.js:227:8)
    at Assertion.be (packages/kbn-expect/expect.js:69:22)
    at Context.it (test/functional/apps/discover/_errors.js:41:40)

Kibana Pipeline / kibana-intake-agent / Jest Integration Tests.packages/kbn-plugin-generator/integration_tests.running the plugin-generator via 'node scripts/generate_plugin.js plugin-name' with default config then running with es instance 'yarn start' should result in the spec plugin being initialized on kibana's stdout

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has failed 4 times on tracked branches: https://github.com/elastic/kibana/issues/54481

Stack Trace

Error: Command failed with exit code 126: ./bin/elasticsearch-keystore create
    at makeError (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/execa/lib/error.js:56:11)
    at handlePromise (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/execa/index.js:114:26)
    at process._tickCallback (internal/process/next_tick.js:68:7)

History

💛 Build #34050 was flaky 16bbeef
💚 Build #34049 succeeded 2b3864f
💔 Build #33739 failed 2444ebf
💔 Build #33231 failed 9ca7644
💔 Build #32882 failed df058c2

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

lukasolson added 9 commits December 9, 2019 11:16

Add async search strategy

02c57f3

Add async search

41b7c6a

Fix async strategy and add tests

cf5ceb9

Merge branch 'master' into asyncSearch

5ad5b36

Move types to separate file

2009343

Revert changes to demo search

ebebdaf

Merge branch 'master' into asyncSearch

65c39b6

Update demo search strategy to use async

5dd876f

Add async es search strategy

d487812

lukasolson added Feature:Search Querying infrastructure in Kibana WIP Work in progress Team:AppArch labels Dec 18, 2019

lukasolson self-assigned this Dec 18, 2019

lukasolson added 2 commits December 18, 2019 17:09

Return response as rawResponse

7433aa6

Poll after initial request

93ea93e

lukasolson mentioned this pull request Jan 13, 2020

Users selection to allow queries to run beyond configurable timeout #24227

Closed

lukasolson added 2 commits January 15, 2020 14:41

Merge branch 'master' into asyncSearch

8e8eb7b

Merge branch 'master' into enhancedEsSearch

f15c4af

lukasolson mentioned this pull request Jan 17, 2020

[Search service] Add sort parameters for async search requests #55141

Closed

lukasolson added 9 commits January 23, 2020 09:44

Merge branch 'master' into asyncSearch

50b4b69

Merge branch 'master' into asyncSearch

2bed4a6

Merge branch 'asyncSearch' into enhancedEsSearch

b6d9c25

Add cancellation to search strategies

7260761

Add tests

be300db

Merge branch 'master' into asyncSearch

fa02de6

Merge branch 'asyncSearch' into enhancedEsSearch

4e524d1

Merge branch 'master' into asyncSearch

7ad5755

Simplify async search strategy

e4eb651

lukasolson mentioned this pull request Mar 9, 2020

[skip-ci] UI for cancelling queries & running beyond timeout #59714

Closed

7 tasks

lukasolson changed the title ~~[skip-ci] [Search service] Asynchronous ES search strategy~~ [Search service] Asynchronous ES search strategy Mar 10, 2020

lukasolson removed the WIP Work in progress label Mar 10, 2020

lukasolson added 3 commits March 10, 2020 09:51

Merge branch 'master' into enhancedEsSearch

5023a5e

Merge branch 'master' into enhancedEsSearch

bc96081

Fix merge error

3976cdb

lizozom reviewed Mar 12, 2020

View reviewed changes

lizozom approved these changes Mar 12, 2020

View reviewed changes

Review feedback

df058c2

lizozom mentioned this pull request Mar 12, 2020

Async search cancellation UI #60058

Closed

7 tasks

lukasolson added 2 commits March 13, 2020 16:48

Fix issue with hits.total

05ab470

Merge branch 'master' into enhancedEsSearch

9ca7644

Merge branch 'master' into enhancedEsSearch

2444ebf

Merge branch 'master' into enhancedEsSearch

2b3864f

Merge branch 'master' into enhancedEsSearch

16bbeef

lukasolson merged commit ac5e323 into elastic:master Mar 18, 2020

lukasolson mentioned this pull request Mar 18, 2020

[7.x] [Search service] Asynchronous ES search strategy (#53538) #60469

Merged

kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label Mar 19, 2020

kibanamachine removed the backport missing Added to PRs automatically when the are determined to be missing a backport. label Mar 20, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Search service] Asynchronous ES search strategy #53538

[Search service] Asynchronous ES search strategy #53538

lukasolson commented Dec 18, 2019 •

edited by jinmu03

Loading

elasticmachine commented Dec 18, 2019

lizozom Mar 12, 2020

lukasolson Mar 17, 2020

lizozom left a comment

lizozom commented Mar 17, 2020

lizozom commented Mar 17, 2020

lukasolson commented Mar 17, 2020

kibanamachine commented Mar 19, 2020

kibanamachine commented Mar 20, 2020

kibanamachine commented Apr 10, 2020

Standard Out

Stack Trace

Standard Out

Stack Trace

Standard Out

Stack Trace

[Search service] Asynchronous ES search strategy #53538

[Search service] Asynchronous ES search strategy #53538

Conversation

lukasolson commented Dec 18, 2019 • edited by jinmu03 Loading

Dev Docs

To do

elasticmachine commented Dec 18, 2019

lizozom Mar 12, 2020

Choose a reason for hiding this comment

lukasolson Mar 17, 2020

Choose a reason for hiding this comment

lizozom left a comment

Choose a reason for hiding this comment

lizozom commented Mar 17, 2020

lizozom commented Mar 17, 2020

lukasolson commented Mar 17, 2020

kibanamachine commented Mar 19, 2020

kibanamachine commented Mar 20, 2020

kibanamachine commented Apr 10, 2020

💔 Build Failed

Test Failures

Standard Out

Stack Trace

Standard Out

Stack Trace

Standard Out

Stack Trace

History

lukasolson commented Dec 18, 2019 •

edited by jinmu03

Loading