-
Notifications
You must be signed in to change notification settings - Fork 8.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Ingest Manager] Plugin won't retry failed initial setup #74587
Labels
Team:Fleet
Team label for Observability Data Collection Fleet team
Comments
thomasneirynck
added
Team:Fleet
Team label for Observability Data Collection Fleet team
triage_needed
labels
Aug 6, 2020
Pinging @elastic/ingest-management (Team:Ingest Management) |
This was referenced Aug 6, 2020
1 task
jfsiii
pushed a commit
that referenced
this issue
Aug 20, 2020
* add retries for registry requests. works, afaict. no tests. one TS issue. * Fix TS issue. Add link to node-fetch error docs * Restore some accidentally deleted code. * Add more comments. Remove logging. * Add tests for plugin setup service & handlers * Add tests for Registry retry logic * Extract setup retry logic to separate function/file * Add tests for setup retry logic ``` firstSuccessOrTryAgain ✓ reject/throws is called again & its value returned (18ms) ✓ the first success value is cached (2ms) ``` * More straightforward(?) tests for setup caching * Revert cached setup. Still limit 1 call at a time Terrible tests. Committing & pushing to see if it fixes failures like https://github.com/elastic/kibana/pull/74507/checks?check_run_id=980178887 https://kibana-ci.elastic.co/job/elastic+kibana+pipeline-pull-request/67892/execution/node/663/log/ ``` 07:36:56 └-> "before all" hook 07:36:56 └-> should not allow to enroll an agent with a invalid enrollment 07:36:56 └-> "before each" hook: global before each 07:36:56 └-> "before each" hook: beforeSetupWithDockerRegistry 07:36:56 │ proc [kibana] error [11:36:56.369] Error: Internal Server Error 07:36:56 │ proc [kibana] at HapiResponseAdapter.toError (/dev/shm/workspace/parallel/5/kibana/build/kibana-build-xpack/src/core/server/http/router/response_adapter.js:132:19) 07:36:56 │ proc [kibana] at HapiResponseAdapter.toHapiResponse (/dev/shm/workspace/parallel/5/kibana/build/kibana-build-xpack/src/core/server/http/router/response_adapter.js:86:19) 07:36:56 │ proc [kibana] at HapiResponseAdapter.handle (/dev/shm/workspace/parallel/5/kibana/build/kibana-build-xpack/src/core/server/http/router/response_adapter.js:81:17) 07:36:56 │ proc [kibana] at Router.handle (/dev/shm/workspace/parallel/5/kibana/build/kibana-build-xpack/src/core/server/http/router/router.js:164:34) 07:36:56 │ proc [kibana] at process._tickCallback (internal/process/next_tick.js:68:7) 07:36:56 │ proc [kibana] log [11:36:56.581] [info][authentication][plugins][security] Authentication attempt failed: [security_exception] missing authentication credentials for REST request [/_security/_authenticate], with { header={ WWW-Authenticate={ 0="ApiKey" & 1="Basic realm=\"security\" charset=\"UTF-8\"" } } } 07:36:56 └- ✓ pass (60ms) "Ingest Manager Endpoints Fleet Endpoints fleet_agents_enroll should not allow to enroll an agent with a invalid enrollment" 07:36:56 └-> should not allow to enroll an agent with a shared id if it already exists 07:36:56 └-> "before each" hook: global before each 07:36:56 └-> "before each" hook: beforeSetupWithDockerRegistry 07:36:56 └- ✓ pass (111ms) "Ingest Manager Endpoints Fleet Endpoints fleet_agents_enroll should not allow to enroll an agent with a shared id if it already exists " 07:36:56 └-> should not allow to enroll an agent with a version > kibana 07:36:56 └-> "before each" hook: global before each 07:36:56 └-> "before each" hook: beforeSetupWithDockerRegistry 07:36:56 └- ✓ pass (58ms) "Ingest Manager Endpoints Fleet Endpoints fleet_agents_enroll should not allow to enroll an agent with a version > kibana" 07:36:56 └-> should allow to enroll an agent with a valid enrollment token 07:36:56 └-> "before each" hook: global before each 07:36:56 └-> "before each" hook: beforeSetupWithDockerRegistry 07:36:56 └- ✖ fail: Ingest Manager Endpoints Fleet Endpoints fleet_agents_enroll should allow to enroll an agent with a valid enrollment token 07:36:56 │ Error: expected 200 "OK", got 500 "Internal Server Error" 07:36:56 │ at Test._assertStatus (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:268:12) 07:36:56 │ at Test._assertFunction (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:283:11) 07:36:56 │ at Test.assert (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:173:18) 07:36:56 │ at assert (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:131:12) 07:36:56 │ at /dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:128:5 07:36:56 │ at Test.Request.callback (/dev/shm/workspace/kibana/node_modules/superagent/lib/node/index.js:718:3) 07:36:56 │ at parser (/dev/shm/workspace/kibana/node_modules/superagent/lib/node/index.js:906:18) 07:36:56 │ at IncomingMessage.res.on (/dev/shm/workspace/kibana/node_modules/superagent/lib/node/parsers/json.js:19:7) 07:36:56 │ at endReadableNT (_stream_readable.js:1145:12) 07:36:56 │ at process._tickCallback (internal/process/next_tick.js:63:19) 07:36:56 │ 07:36:56 │ ``` * New name & tests for one-at-a-time /setup behavior `firstPromiseBlocksAndFufills` for "the first promise created blocks others from being created, then fufills all with that first result" * More (better?) renaming * Fix name in test description * Fix spelling typo. * Remove registry retry code & tests * Use async fn's .catch to avoid unhandled rejection Add explicit `isPending` value instead of overloading role of `status`. Could probably do without it, but it makes the intent more clear. Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
jfsiii
pushed a commit
to jfsiii/kibana
that referenced
this issue
Aug 20, 2020
elastic#75372) * add retries for registry requests. works, afaict. no tests. one TS issue. * Fix TS issue. Add link to node-fetch error docs * Restore some accidentally deleted code. * Add more comments. Remove logging. * Add tests for plugin setup service & handlers * Add tests for Registry retry logic * Extract setup retry logic to separate function/file * Add tests for setup retry logic ``` firstSuccessOrTryAgain ✓ reject/throws is called again & its value returned (18ms) ✓ the first success value is cached (2ms) ``` * More straightforward(?) tests for setup caching * Revert cached setup. Still limit 1 call at a time Terrible tests. Committing & pushing to see if it fixes failures like https://github.com/elastic/kibana/pull/74507/checks?check_run_id=980178887 https://kibana-ci.elastic.co/job/elastic+kibana+pipeline-pull-request/67892/execution/node/663/log/ ``` 07:36:56 └-> "before all" hook 07:36:56 └-> should not allow to enroll an agent with a invalid enrollment 07:36:56 └-> "before each" hook: global before each 07:36:56 └-> "before each" hook: beforeSetupWithDockerRegistry 07:36:56 │ proc [kibana] error [11:36:56.369] Error: Internal Server Error 07:36:56 │ proc [kibana] at HapiResponseAdapter.toError (/dev/shm/workspace/parallel/5/kibana/build/kibana-build-xpack/src/core/server/http/router/response_adapter.js:132:19) 07:36:56 │ proc [kibana] at HapiResponseAdapter.toHapiResponse (/dev/shm/workspace/parallel/5/kibana/build/kibana-build-xpack/src/core/server/http/router/response_adapter.js:86:19) 07:36:56 │ proc [kibana] at HapiResponseAdapter.handle (/dev/shm/workspace/parallel/5/kibana/build/kibana-build-xpack/src/core/server/http/router/response_adapter.js:81:17) 07:36:56 │ proc [kibana] at Router.handle (/dev/shm/workspace/parallel/5/kibana/build/kibana-build-xpack/src/core/server/http/router/router.js:164:34) 07:36:56 │ proc [kibana] at process._tickCallback (internal/process/next_tick.js:68:7) 07:36:56 │ proc [kibana] log [11:36:56.581] [info][authentication][plugins][security] Authentication attempt failed: [security_exception] missing authentication credentials for REST request [/_security/_authenticate], with { header={ WWW-Authenticate={ 0="ApiKey" & 1="Basic realm=\"security\" charset=\"UTF-8\"" } } } 07:36:56 └- ✓ pass (60ms) "Ingest Manager Endpoints Fleet Endpoints fleet_agents_enroll should not allow to enroll an agent with a invalid enrollment" 07:36:56 └-> should not allow to enroll an agent with a shared id if it already exists 07:36:56 └-> "before each" hook: global before each 07:36:56 └-> "before each" hook: beforeSetupWithDockerRegistry 07:36:56 └- ✓ pass (111ms) "Ingest Manager Endpoints Fleet Endpoints fleet_agents_enroll should not allow to enroll an agent with a shared id if it already exists " 07:36:56 └-> should not allow to enroll an agent with a version > kibana 07:36:56 └-> "before each" hook: global before each 07:36:56 └-> "before each" hook: beforeSetupWithDockerRegistry 07:36:56 └- ✓ pass (58ms) "Ingest Manager Endpoints Fleet Endpoints fleet_agents_enroll should not allow to enroll an agent with a version > kibana" 07:36:56 └-> should allow to enroll an agent with a valid enrollment token 07:36:56 └-> "before each" hook: global before each 07:36:56 └-> "before each" hook: beforeSetupWithDockerRegistry 07:36:56 └- ✖ fail: Ingest Manager Endpoints Fleet Endpoints fleet_agents_enroll should allow to enroll an agent with a valid enrollment token 07:36:56 │ Error: expected 200 "OK", got 500 "Internal Server Error" 07:36:56 │ at Test._assertStatus (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:268:12) 07:36:56 │ at Test._assertFunction (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:283:11) 07:36:56 │ at Test.assert (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:173:18) 07:36:56 │ at assert (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:131:12) 07:36:56 │ at /dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:128:5 07:36:56 │ at Test.Request.callback (/dev/shm/workspace/kibana/node_modules/superagent/lib/node/index.js:718:3) 07:36:56 │ at parser (/dev/shm/workspace/kibana/node_modules/superagent/lib/node/index.js:906:18) 07:36:56 │ at IncomingMessage.res.on (/dev/shm/workspace/kibana/node_modules/superagent/lib/node/parsers/json.js:19:7) 07:36:56 │ at endReadableNT (_stream_readable.js:1145:12) 07:36:56 │ at process._tickCallback (internal/process/next_tick.js:63:19) 07:36:56 │ 07:36:56 │ ``` * New name & tests for one-at-a-time /setup behavior `firstPromiseBlocksAndFufills` for "the first promise created blocks others from being created, then fufills all with that first result" * More (better?) renaming * Fix name in test description * Fix spelling typo. * Remove registry retry code & tests * Use async fn's .catch to avoid unhandled rejection Add explicit `isPending` value instead of overloading role of `status`. Could probably do without it, but it makes the intent more clear. Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
jfsiii
pushed a commit
to jfsiii/kibana
that referenced
this issue
Aug 20, 2020
elastic#75372) * add retries for registry requests. works, afaict. no tests. one TS issue. * Fix TS issue. Add link to node-fetch error docs * Restore some accidentally deleted code. * Add more comments. Remove logging. * Add tests for plugin setup service & handlers * Add tests for Registry retry logic * Extract setup retry logic to separate function/file * Add tests for setup retry logic ``` firstSuccessOrTryAgain ✓ reject/throws is called again & its value returned (18ms) ✓ the first success value is cached (2ms) ``` * More straightforward(?) tests for setup caching * Revert cached setup. Still limit 1 call at a time Terrible tests. Committing & pushing to see if it fixes failures like https://github.com/elastic/kibana/pull/74507/checks?check_run_id=980178887 https://kibana-ci.elastic.co/job/elastic+kibana+pipeline-pull-request/67892/execution/node/663/log/ ``` 07:36:56 └-> "before all" hook 07:36:56 └-> should not allow to enroll an agent with a invalid enrollment 07:36:56 └-> "before each" hook: global before each 07:36:56 └-> "before each" hook: beforeSetupWithDockerRegistry 07:36:56 │ proc [kibana] error [11:36:56.369] Error: Internal Server Error 07:36:56 │ proc [kibana] at HapiResponseAdapter.toError (/dev/shm/workspace/parallel/5/kibana/build/kibana-build-xpack/src/core/server/http/router/response_adapter.js:132:19) 07:36:56 │ proc [kibana] at HapiResponseAdapter.toHapiResponse (/dev/shm/workspace/parallel/5/kibana/build/kibana-build-xpack/src/core/server/http/router/response_adapter.js:86:19) 07:36:56 │ proc [kibana] at HapiResponseAdapter.handle (/dev/shm/workspace/parallel/5/kibana/build/kibana-build-xpack/src/core/server/http/router/response_adapter.js:81:17) 07:36:56 │ proc [kibana] at Router.handle (/dev/shm/workspace/parallel/5/kibana/build/kibana-build-xpack/src/core/server/http/router/router.js:164:34) 07:36:56 │ proc [kibana] at process._tickCallback (internal/process/next_tick.js:68:7) 07:36:56 │ proc [kibana] log [11:36:56.581] [info][authentication][plugins][security] Authentication attempt failed: [security_exception] missing authentication credentials for REST request [/_security/_authenticate], with { header={ WWW-Authenticate={ 0="ApiKey" & 1="Basic realm=\"security\" charset=\"UTF-8\"" } } } 07:36:56 └- ✓ pass (60ms) "Ingest Manager Endpoints Fleet Endpoints fleet_agents_enroll should not allow to enroll an agent with a invalid enrollment" 07:36:56 └-> should not allow to enroll an agent with a shared id if it already exists 07:36:56 └-> "before each" hook: global before each 07:36:56 └-> "before each" hook: beforeSetupWithDockerRegistry 07:36:56 └- ✓ pass (111ms) "Ingest Manager Endpoints Fleet Endpoints fleet_agents_enroll should not allow to enroll an agent with a shared id if it already exists " 07:36:56 └-> should not allow to enroll an agent with a version > kibana 07:36:56 └-> "before each" hook: global before each 07:36:56 └-> "before each" hook: beforeSetupWithDockerRegistry 07:36:56 └- ✓ pass (58ms) "Ingest Manager Endpoints Fleet Endpoints fleet_agents_enroll should not allow to enroll an agent with a version > kibana" 07:36:56 └-> should allow to enroll an agent with a valid enrollment token 07:36:56 └-> "before each" hook: global before each 07:36:56 └-> "before each" hook: beforeSetupWithDockerRegistry 07:36:56 └- ✖ fail: Ingest Manager Endpoints Fleet Endpoints fleet_agents_enroll should allow to enroll an agent with a valid enrollment token 07:36:56 │ Error: expected 200 "OK", got 500 "Internal Server Error" 07:36:56 │ at Test._assertStatus (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:268:12) 07:36:56 │ at Test._assertFunction (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:283:11) 07:36:56 │ at Test.assert (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:173:18) 07:36:56 │ at assert (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:131:12) 07:36:56 │ at /dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:128:5 07:36:56 │ at Test.Request.callback (/dev/shm/workspace/kibana/node_modules/superagent/lib/node/index.js:718:3) 07:36:56 │ at parser (/dev/shm/workspace/kibana/node_modules/superagent/lib/node/index.js:906:18) 07:36:56 │ at IncomingMessage.res.on (/dev/shm/workspace/kibana/node_modules/superagent/lib/node/parsers/json.js:19:7) 07:36:56 │ at endReadableNT (_stream_readable.js:1145:12) 07:36:56 │ at process._tickCallback (internal/process/next_tick.js:63:19) 07:36:56 │ 07:36:56 │ ``` * New name & tests for one-at-a-time /setup behavior `firstPromiseBlocksAndFufills` for "the first promise created blocks others from being created, then fufills all with that first result" * More (better?) renaming * Fix name in test description * Fix spelling typo. * Remove registry retry code & tests * Use async fn's .catch to avoid unhandled rejection Add explicit `isPending` value instead of overloading role of `status`. Could probably do without it, but it makes the intent more clear. Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> # Conflicts: # x-pack/plugins/ingest_manager/server/services/setup.ts
jfsiii
pushed a commit
that referenced
this issue
Aug 21, 2020
…) (#75583) * add retries for registry requests. works, afaict. no tests. one TS issue. * Fix TS issue. Add link to node-fetch error docs * Restore some accidentally deleted code. * Add more comments. Remove logging. * Add tests for plugin setup service & handlers * Add tests for Registry retry logic * Extract setup retry logic to separate function/file * Add tests for setup retry logic ``` firstSuccessOrTryAgain ✓ reject/throws is called again & its value returned (18ms) ✓ the first success value is cached (2ms) ``` * More straightforward(?) tests for setup caching * Revert cached setup. Still limit 1 call at a time Terrible tests. Committing & pushing to see if it fixes failures like https://github.com/elastic/kibana/pull/74507/checks?check_run_id=980178887 https://kibana-ci.elastic.co/job/elastic+kibana+pipeline-pull-request/67892/execution/node/663/log/ ``` 07:36:56 └-> "before all" hook 07:36:56 └-> should not allow to enroll an agent with a invalid enrollment 07:36:56 └-> "before each" hook: global before each 07:36:56 └-> "before each" hook: beforeSetupWithDockerRegistry 07:36:56 │ proc [kibana] error [11:36:56.369] Error: Internal Server Error 07:36:56 │ proc [kibana] at HapiResponseAdapter.toError (/dev/shm/workspace/parallel/5/kibana/build/kibana-build-xpack/src/core/server/http/router/response_adapter.js:132:19) 07:36:56 │ proc [kibana] at HapiResponseAdapter.toHapiResponse (/dev/shm/workspace/parallel/5/kibana/build/kibana-build-xpack/src/core/server/http/router/response_adapter.js:86:19) 07:36:56 │ proc [kibana] at HapiResponseAdapter.handle (/dev/shm/workspace/parallel/5/kibana/build/kibana-build-xpack/src/core/server/http/router/response_adapter.js:81:17) 07:36:56 │ proc [kibana] at Router.handle (/dev/shm/workspace/parallel/5/kibana/build/kibana-build-xpack/src/core/server/http/router/router.js:164:34) 07:36:56 │ proc [kibana] at process._tickCallback (internal/process/next_tick.js:68:7) 07:36:56 │ proc [kibana] log [11:36:56.581] [info][authentication][plugins][security] Authentication attempt failed: [security_exception] missing authentication credentials for REST request [/_security/_authenticate], with { header={ WWW-Authenticate={ 0="ApiKey" & 1="Basic realm=\"security\" charset=\"UTF-8\"" } } } 07:36:56 └- ✓ pass (60ms) "Ingest Manager Endpoints Fleet Endpoints fleet_agents_enroll should not allow to enroll an agent with a invalid enrollment" 07:36:56 └-> should not allow to enroll an agent with a shared id if it already exists 07:36:56 └-> "before each" hook: global before each 07:36:56 └-> "before each" hook: beforeSetupWithDockerRegistry 07:36:56 └- ✓ pass (111ms) "Ingest Manager Endpoints Fleet Endpoints fleet_agents_enroll should not allow to enroll an agent with a shared id if it already exists " 07:36:56 └-> should not allow to enroll an agent with a version > kibana 07:36:56 └-> "before each" hook: global before each 07:36:56 └-> "before each" hook: beforeSetupWithDockerRegistry 07:36:56 └- ✓ pass (58ms) "Ingest Manager Endpoints Fleet Endpoints fleet_agents_enroll should not allow to enroll an agent with a version > kibana" 07:36:56 └-> should allow to enroll an agent with a valid enrollment token 07:36:56 └-> "before each" hook: global before each 07:36:56 └-> "before each" hook: beforeSetupWithDockerRegistry 07:36:56 └- ✖ fail: Ingest Manager Endpoints Fleet Endpoints fleet_agents_enroll should allow to enroll an agent with a valid enrollment token 07:36:56 │ Error: expected 200 "OK", got 500 "Internal Server Error" 07:36:56 │ at Test._assertStatus (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:268:12) 07:36:56 │ at Test._assertFunction (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:283:11) 07:36:56 │ at Test.assert (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:173:18) 07:36:56 │ at assert (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:131:12) 07:36:56 │ at /dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:128:5 07:36:56 │ at Test.Request.callback (/dev/shm/workspace/kibana/node_modules/superagent/lib/node/index.js:718:3) 07:36:56 │ at parser (/dev/shm/workspace/kibana/node_modules/superagent/lib/node/index.js:906:18) 07:36:56 │ at IncomingMessage.res.on (/dev/shm/workspace/kibana/node_modules/superagent/lib/node/parsers/json.js:19:7) 07:36:56 │ at endReadableNT (_stream_readable.js:1145:12) 07:36:56 │ at process._tickCallback (internal/process/next_tick.js:63:19) 07:36:56 │ 07:36:56 │ ``` * New name & tests for one-at-a-time /setup behavior `firstPromiseBlocksAndFufills` for "the first promise created blocks others from being created, then fufills all with that first result" * More (better?) renaming * Fix name in test description * Fix spelling typo. * Remove registry retry code & tests * Use async fn's .catch to avoid unhandled rejection Add explicit `isPending` value instead of overloading role of `status`. Could probably do without it, but it makes the intent more clear. Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
thomasneirynck
pushed a commit
to thomasneirynck/kibana
that referenced
this issue
Aug 21, 2020
elastic#75372) * add retries for registry requests. works, afaict. no tests. one TS issue. * Fix TS issue. Add link to node-fetch error docs * Restore some accidentally deleted code. * Add more comments. Remove logging. * Add tests for plugin setup service & handlers * Add tests for Registry retry logic * Extract setup retry logic to separate function/file * Add tests for setup retry logic ``` firstSuccessOrTryAgain ✓ reject/throws is called again & its value returned (18ms) ✓ the first success value is cached (2ms) ``` * More straightforward(?) tests for setup caching * Revert cached setup. Still limit 1 call at a time Terrible tests. Committing & pushing to see if it fixes failures like https://github.com/elastic/kibana/pull/74507/checks?check_run_id=980178887 https://kibana-ci.elastic.co/job/elastic+kibana+pipeline-pull-request/67892/execution/node/663/log/ ``` 07:36:56 └-> "before all" hook 07:36:56 └-> should not allow to enroll an agent with a invalid enrollment 07:36:56 └-> "before each" hook: global before each 07:36:56 └-> "before each" hook: beforeSetupWithDockerRegistry 07:36:56 │ proc [kibana] error [11:36:56.369] Error: Internal Server Error 07:36:56 │ proc [kibana] at HapiResponseAdapter.toError (/dev/shm/workspace/parallel/5/kibana/build/kibana-build-xpack/src/core/server/http/router/response_adapter.js:132:19) 07:36:56 │ proc [kibana] at HapiResponseAdapter.toHapiResponse (/dev/shm/workspace/parallel/5/kibana/build/kibana-build-xpack/src/core/server/http/router/response_adapter.js:86:19) 07:36:56 │ proc [kibana] at HapiResponseAdapter.handle (/dev/shm/workspace/parallel/5/kibana/build/kibana-build-xpack/src/core/server/http/router/response_adapter.js:81:17) 07:36:56 │ proc [kibana] at Router.handle (/dev/shm/workspace/parallel/5/kibana/build/kibana-build-xpack/src/core/server/http/router/router.js:164:34) 07:36:56 │ proc [kibana] at process._tickCallback (internal/process/next_tick.js:68:7) 07:36:56 │ proc [kibana] log [11:36:56.581] [info][authentication][plugins][security] Authentication attempt failed: [security_exception] missing authentication credentials for REST request [/_security/_authenticate], with { header={ WWW-Authenticate={ 0="ApiKey" & 1="Basic realm=\"security\" charset=\"UTF-8\"" } } } 07:36:56 └- ✓ pass (60ms) "Ingest Manager Endpoints Fleet Endpoints fleet_agents_enroll should not allow to enroll an agent with a invalid enrollment" 07:36:56 └-> should not allow to enroll an agent with a shared id if it already exists 07:36:56 └-> "before each" hook: global before each 07:36:56 └-> "before each" hook: beforeSetupWithDockerRegistry 07:36:56 └- ✓ pass (111ms) "Ingest Manager Endpoints Fleet Endpoints fleet_agents_enroll should not allow to enroll an agent with a shared id if it already exists " 07:36:56 └-> should not allow to enroll an agent with a version > kibana 07:36:56 └-> "before each" hook: global before each 07:36:56 └-> "before each" hook: beforeSetupWithDockerRegistry 07:36:56 └- ✓ pass (58ms) "Ingest Manager Endpoints Fleet Endpoints fleet_agents_enroll should not allow to enroll an agent with a version > kibana" 07:36:56 └-> should allow to enroll an agent with a valid enrollment token 07:36:56 └-> "before each" hook: global before each 07:36:56 └-> "before each" hook: beforeSetupWithDockerRegistry 07:36:56 └- ✖ fail: Ingest Manager Endpoints Fleet Endpoints fleet_agents_enroll should allow to enroll an agent with a valid enrollment token 07:36:56 │ Error: expected 200 "OK", got 500 "Internal Server Error" 07:36:56 │ at Test._assertStatus (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:268:12) 07:36:56 │ at Test._assertFunction (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:283:11) 07:36:56 │ at Test.assert (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:173:18) 07:36:56 │ at assert (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:131:12) 07:36:56 │ at /dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:128:5 07:36:56 │ at Test.Request.callback (/dev/shm/workspace/kibana/node_modules/superagent/lib/node/index.js:718:3) 07:36:56 │ at parser (/dev/shm/workspace/kibana/node_modules/superagent/lib/node/index.js:906:18) 07:36:56 │ at IncomingMessage.res.on (/dev/shm/workspace/kibana/node_modules/superagent/lib/node/parsers/json.js:19:7) 07:36:56 │ at endReadableNT (_stream_readable.js:1145:12) 07:36:56 │ at process._tickCallback (internal/process/next_tick.js:63:19) 07:36:56 │ 07:36:56 │ ``` * New name & tests for one-at-a-time /setup behavior `firstPromiseBlocksAndFufills` for "the first promise created blocks others from being created, then fufills all with that first result" * More (better?) renaming * Fix name in test description * Fix spelling typo. * Remove registry retry code & tests * Use async fn's .catch to avoid unhandled rejection Add explicit `isPending` value instead of overloading role of `status`. Could probably do without it, but it makes the intent more clear. Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
jfsiii
pushed a commit
that referenced
this issue
Aug 22, 2020
…) (#75587) * add retries for registry requests. works, afaict. no tests. one TS issue. * Fix TS issue. Add link to node-fetch error docs * Restore some accidentally deleted code. * Add more comments. Remove logging. * Add tests for plugin setup service & handlers * Add tests for Registry retry logic * Extract setup retry logic to separate function/file * Add tests for setup retry logic ``` firstSuccessOrTryAgain ✓ reject/throws is called again & its value returned (18ms) ✓ the first success value is cached (2ms) ``` * More straightforward(?) tests for setup caching * Revert cached setup. Still limit 1 call at a time Terrible tests. Committing & pushing to see if it fixes failures like https://github.com/elastic/kibana/pull/74507/checks?check_run_id=980178887 https://kibana-ci.elastic.co/job/elastic+kibana+pipeline-pull-request/67892/execution/node/663/log/ ``` 07:36:56 └-> "before all" hook 07:36:56 └-> should not allow to enroll an agent with a invalid enrollment 07:36:56 └-> "before each" hook: global before each 07:36:56 └-> "before each" hook: beforeSetupWithDockerRegistry 07:36:56 │ proc [kibana] error [11:36:56.369] Error: Internal Server Error 07:36:56 │ proc [kibana] at HapiResponseAdapter.toError (/dev/shm/workspace/parallel/5/kibana/build/kibana-build-xpack/src/core/server/http/router/response_adapter.js:132:19) 07:36:56 │ proc [kibana] at HapiResponseAdapter.toHapiResponse (/dev/shm/workspace/parallel/5/kibana/build/kibana-build-xpack/src/core/server/http/router/response_adapter.js:86:19) 07:36:56 │ proc [kibana] at HapiResponseAdapter.handle (/dev/shm/workspace/parallel/5/kibana/build/kibana-build-xpack/src/core/server/http/router/response_adapter.js:81:17) 07:36:56 │ proc [kibana] at Router.handle (/dev/shm/workspace/parallel/5/kibana/build/kibana-build-xpack/src/core/server/http/router/router.js:164:34) 07:36:56 │ proc [kibana] at process._tickCallback (internal/process/next_tick.js:68:7) 07:36:56 │ proc [kibana] log [11:36:56.581] [info][authentication][plugins][security] Authentication attempt failed: [security_exception] missing authentication credentials for REST request [/_security/_authenticate], with { header={ WWW-Authenticate={ 0="ApiKey" & 1="Basic realm=\"security\" charset=\"UTF-8\"" } } } 07:36:56 └- ✓ pass (60ms) "Ingest Manager Endpoints Fleet Endpoints fleet_agents_enroll should not allow to enroll an agent with a invalid enrollment" 07:36:56 └-> should not allow to enroll an agent with a shared id if it already exists 07:36:56 └-> "before each" hook: global before each 07:36:56 └-> "before each" hook: beforeSetupWithDockerRegistry 07:36:56 └- ✓ pass (111ms) "Ingest Manager Endpoints Fleet Endpoints fleet_agents_enroll should not allow to enroll an agent with a shared id if it already exists " 07:36:56 └-> should not allow to enroll an agent with a version > kibana 07:36:56 └-> "before each" hook: global before each 07:36:56 └-> "before each" hook: beforeSetupWithDockerRegistry 07:36:56 └- ✓ pass (58ms) "Ingest Manager Endpoints Fleet Endpoints fleet_agents_enroll should not allow to enroll an agent with a version > kibana" 07:36:56 └-> should allow to enroll an agent with a valid enrollment token 07:36:56 └-> "before each" hook: global before each 07:36:56 └-> "before each" hook: beforeSetupWithDockerRegistry 07:36:56 └- ✖ fail: Ingest Manager Endpoints Fleet Endpoints fleet_agents_enroll should allow to enroll an agent with a valid enrollment token 07:36:56 │ Error: expected 200 "OK", got 500 "Internal Server Error" 07:36:56 │ at Test._assertStatus (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:268:12) 07:36:56 │ at Test._assertFunction (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:283:11) 07:36:56 │ at Test.assert (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:173:18) 07:36:56 │ at assert (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:131:12) 07:36:56 │ at /dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:128:5 07:36:56 │ at Test.Request.callback (/dev/shm/workspace/kibana/node_modules/superagent/lib/node/index.js:718:3) 07:36:56 │ at parser (/dev/shm/workspace/kibana/node_modules/superagent/lib/node/index.js:906:18) 07:36:56 │ at IncomingMessage.res.on (/dev/shm/workspace/kibana/node_modules/superagent/lib/node/parsers/json.js:19:7) 07:36:56 │ at endReadableNT (_stream_readable.js:1145:12) 07:36:56 │ at process._tickCallback (internal/process/next_tick.js:63:19) 07:36:56 │ 07:36:56 │ ``` * New name & tests for one-at-a-time /setup behavior `firstPromiseBlocksAndFufills` for "the first promise created blocks others from being created, then fufills all with that first result" * More (better?) renaming * Fix name in test description * Fix spelling typo. * Remove registry retry code & tests * Use async fn's .catch to avoid unhandled rejection Add explicit `isPending` value instead of overloading role of `status`. Could probably do without it, but it makes the intent more clear. Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> # Conflicts: # x-pack/plugins/ingest_manager/server/services/setup.ts Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Steps to reproduce:
curl -X POST 'http://localhost:5601/api/ingest_manager/setup' -H 'kbn-xsrf: <string>' --user elastic:changeme
{"statusCode":502,"error":"Bad Gateway","message":"Error connecting to package registry at http://localhost:8080/package/system/0.5.3: request to http://localhost:8080/package/system/0.5.3 failed, reason: connect ECONNREFUSED 127.0.0.1:8080"}
curl http://localhost:5601/api/ingest_manager/epm/packages --user elastic:changeme
Movies showing Ingest manager setup retains first result.zip
Root cause
We hold onto the value in
setupIngestStatus
Promise to prevent simultaneous calls to setup. However, when we reject the promise due to a failure that means we'll never retry since the nextawait setupIngestStatus
will failkibana/x-pack/plugins/ingest_manager/server/services/setup.ts
Lines 128 to 134 in beb7b82
The text was updated successfully, but these errors were encountered: