Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failing test: Chrome X-Pack UI Functional Tests.x-pack/test/functional_with_es_ssl/apps/triggers_actions_ui/details·ts - Actions and Triggers app Alert Details Alert Instances renders the active alert instances #57426

Closed
kibanamachine opened this issue Feb 12, 2020 · 32 comments · Fixed by #57506, #58994, #60893 or #83478
Closed

Failing test: Chrome X-Pack UI Functional Tests.x-pack/test/functional_with_es_ssl/apps/triggers_actions_ui/details·ts - Actions and Triggers app Alert Details Alert Instances renders the active alert instances #57426

kibanamachine opened this issue Feb 12, 2020 · 32 comments · Fixed by #57506, #58994, #60893 or #83478
Assignees
@ymao1
Labels
blocker failed-test A test failure on a tracked branch, potentially flaky-test Feature:Alerting skipped-test Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) v7.11.0 v8.0.0

Comments

@kibanamachine
Copy link
Contributor

kibanamachine commented Feb 12, 2020 •
edited
Loading

A test failed on a tracked branch

{ Error: expected [ { instance: 'us-central',
    status: 'Active',
    start: '12 Feb 2020 @ 08:08:30' },
  { instance: 'us-east',
    status: 'Active',
    start: '12 Feb 2020 @ 08:08:30' },
  { instance: 'us-west',
    status: 'Active',
    start: '12 Feb 2020 @ 08:08:31' } ] to sort of equal [ { instance: 'us-central',
    status: 'Active',
    start: '12 Feb 2020 @ 08:08:30' },
  { instance: 'us-east',
    status: 'Active',
    start: '12 Feb 2020 @ 08:08:30' },
  { instance: 'us-west',
    status: 'Active',
    start: '12 Feb 2020 @ 08:08:30' } ]
    at Assertion.assert (/dev/shm/workspace/kibana/packages/kbn-expect/expect.js:100:11)
    at Assertion.eql (/dev/shm/workspace/kibana/packages/kbn-expect/expect.js:244:8)
    at Context.it (test/functional_with_es_ssl/apps/triggers_actions_ui/details.ts:228:78)
    at process._tickCallback (internal/process/next_tick.js:68:7)
  actual:
   '[\n  {\n    "instance": "us-central"\n    "start": "12 Feb 2020 @ 08:08:30"\n    "status": "Active"\n  }\n  {\n    "instance": "us-east"\n    "start": "12 Feb 2020 @ 08:08:30"\n    "status": "Active"\n  }\n  {\n    "instance": "us-west"\n    "start": "12 Feb 2020 @ 08:08:31"\n    "status": "Active"\n  }\n]',
  expected:
   '[\n  {\n    "instance": "us-central"\n    "start": "12 Feb 2020 @ 08:08:30"\n    "status": "Active"\n  }\n  {\n    "instance": "us-east"\n    "start": "12 Feb 2020 @ 08:08:30"\n    "status": "Active"\n  }\n  {\n    "instance": "us-west"\n    "start": "12 Feb 2020 @ 08:08:30"\n    "status": "Active"\n  }\n]',
  showDiff: true }

First failure: Jenkins Build
@kibanamachine kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Feb 12, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-test-triage (failed-test)

@gmmorris gmmorris self-assigned this Feb 12, 2020
@gmmorris gmmorris added Feature:Alerting Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) labels Feb 12, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-alerting-services (Team:Alerting Services)

@gmmorris gmmorris mentioned this issue Feb 12, 2020
Merged
7 tasks
@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

1 similar comment
@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

@kibanamachine kibanamachine reopened this Feb 13, 2020
@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

1 similar comment
@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

@gmmorris
Copy link
Contributor

Should be addressed now as backport has finally made it through CI

@kibanamachine kibanamachine reopened this Feb 14, 2020
@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

@mikecote mikecote mentioned this issue Feb 14, 2020
Merged
@kibanamachine kibanamachine reopened this Nov 7, 2020
@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

mistic added a commit that referenced this issue Nov 9, 2020
@mistic
skip flaky suite (#57426)
d1ef0d6
mistic added a commit that referenced this issue Nov 9, 2020
@mistic
skip flaky suite (#57426)
adf7db6
@mistic
Copy link
Member

mistic commented Nov 9, 2020

That test has been failing again both on master and 7.x. Skipping for now

master: d1ef0d6
7.x: adf7db6

gmmorris added a commit to gmmorris/kibana that referenced this issue Nov 9, 2020
@gmmorris
Merge branch 'master' into actions/license-allowlist
c2eaff4 
* master:
  Added `defaultActionMessage` to index threshold alert UI type definition (elastic#80936)
  [ILM] Migrate Delete phase and name field to Form Lib (elastic#82834)
  skip flaky suite (elastic#57426)
  [Alerting] adds an Run When field in the alert flyout to assign the action to an Action Group (elastic#82472)
  [APM] Expose APM event client as part of plugin contract (elastic#82724)
  [Logs UI] Fix errors during navigation (elastic#78319)
  Enable send to background in TSVB (elastic#82835)
  SavedObjects search_dsl: add match_phrase_prefix clauses when using prefix search (elastic#82693)
  [Ingest Manager] Unify install* under installPackage (elastic#82916)
phillipb added a commit to phillipb/kibana that referenced this issue Nov 10, 2020
@phillipb
Merge branch 'master' of github.com:elastic/kibana into bootstrap-nod…
864b5bd 
…e-details-overlay

* 'master' of github.com:elastic/kibana: (201 commits)
  Added `defaultActionMessage` to index threshold alert UI type definition (elastic#80936)
  [ILM] Migrate Delete phase and name field to Form Lib (elastic#82834)
  skip flaky suite (elastic#57426)
  [Alerting] adds an Run When field in the alert flyout to assign the action to an Action Group (elastic#82472)
  [APM] Expose APM event client as part of plugin contract (elastic#82724)
  [Logs UI] Fix errors during navigation (elastic#78319)
  Enable send to background in TSVB (elastic#82835)
  SavedObjects search_dsl: add match_phrase_prefix clauses when using prefix search (elastic#82693)
  [Ingest Manager] Unify install* under installPackage (elastic#82916)
  [Fleet] Make stream id unique in agent policy (elastic#82447)
  skip flaky suite (elastic#82915)
  skip flaky suite (elastic#75794)
  Copy `dateAsStringRt` to observability plugin (elastic#82839)
  [Maps] rename connected_components/map folder to mb_map (elastic#82897)
  [Security Solution] Fix EventsViewer DnD cypress tests (elastic#82619)
  [Security Solution] Adds logging and performance fan out API for threat/Indicator matching (elastic#82546)
  Implemented Alerting health status pusher by using task manager and status pooler for Kibana status plugins 'kibanahost/api/status' (elastic#79056)
  [APM] Adds new configuration 'xpack.apm.maxServiceEnvironments' (elastic#82090)
  Move single use function in line (elastic#82885)
  [ML] Add unsigned_long support to data frame analytics and anomaly detection (elastic#82636)
  ...
@ymao1 ymao1 self-assigned this Nov 11, 2020
@ymao1
Copy link
Contributor

ymao1 commented Nov 12, 2020 •
edited
Loading

I ran this test that failed Alert Details Alert Instances renders the active alert instances through the flaky test runner 100 times and it succeeded every time.

@ymao1 ymao1 closed this as completed Nov 12, 2020
@ymao1 ymao1 reopened this Nov 12, 2020
@pmuellr
Copy link
Member

pmuellr commented Nov 12, 2020

Hmm the failure reported was that it was off by a second on one of the array elements, way back here: #57426 (comment)

Looks like the test is here:

kibana/x-pack/test/functional_with_es_ssl/apps/triggers_actions_ui/details.ts

Lines 393 to 400 in 208e86e

expect(instancesList.map((instance) => omit(instance, 'duration'))).to.eql([
{
instance: 'us-central',
status: 'Active (Default)',
start: moment(dateOnAllInstancesFromApiResponse['us-central'])
.utc()
.format('D MMM YYYY @ HH:mm:ss'),
},

I'm guessing this may be a rounding difference on the millis from the moment() call.

Since rounding may end up affecting even the year (on New Year's Eve :-) ), if it is rounding, I think we'll need to do this test another way. Perhaps parse what the UI is returning, converting to a Date() object, to get the epoch millis value. Then do a compare of the times from the API with a few seconds of slop (eg, 10).

I didn't look to see how the UI is actually calculating this value. Are they both the same? So we shouldn't be seeing the rounding error? Perhaps what's happening is there is some delay, and the alert got another chance to run, and it's a 1 sec interval so it caught the next one?

If so, I think the change suggested above ^^^, taking into account some slop, should still be a safe fix.

@ymao1 ymao1 mentioned this issue Nov 16, 2020
Merged
@kobelb kobelb added the needs-team Issues missing a team label label Jan 31, 2022
@botelastic botelastic bot removed the needs-team Issues missing a team label label Jan 31, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ymao1 ymao1

Labels
blocker failed-test A test failure on a tracked branch, potentially flaky-test Feature:Alerting skipped-test Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) v7.11.0 v8.0.0
Projects
None yet
Milestone
No milestone
9 participants
@pmuellr @gmmorris @kobelb @mikecote @mistic @dmlemeshko @ymao1 @elasticmachine @kibanamachine