Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failing test: Security Solution Cypress.x-pack/test/security_solution_cypress/cypress/e2e/investigations/threat_intelligence/indicators·cy·ts - Multiple indicators Indicator page search should handle all search actions should handle all search actions #195804

Open
kibanamachine opened this issue Oct 10, 2024 · 14 comments
Labels
blocker failed-test A test failure on a tracked branch, potentially flaky-test skipped-test Team:Threat Hunting:Investigations Security Solution Investigations Team v9.0.0

Comments

@kibanamachine
Copy link
Contributor

kibanamachine commented Oct 10, 2024

A test failed on a tracked branch

CypressError: Timed out retrying after 5000ms: `cy.wait()` timed out waiting `5000ms` for the 1st request to the route: `search`. No request ever occurred.

https://on.cypress.io/wait
    at cypressErr (http://localhost:5617/__cypress/runner/cypress_runner.js:75257:18)
    at Object.errByPath (http://localhost:5617/__cypress/runner/cypress_runner.js:75312:10)
    at checkForXhr (http://localhost:5617/__cypress/runner/cypress_runner.js:135500:84)
    at <unknown> (http://localhost:5617/__cypress/runner/cypress_runner.js:135526:28)
    at tryCatcher (http://localhost:5617/__cypress/runner/cypress_runner.js:1807:23)
    at Promise.attempt.Promise.try (http://localhost:5617/__cypress/runner/cypress_runner.js:4315:29)
    at whenStable (http://localhost:5617/__cypress/runner/cypress_runner.js:143918:68)
    at <unknown> (http://localhost:5617/__cypress/runner/cypress_runner.js:143859:14)
    at tryCatcher (http://localhost:5617/__cypress/runner/cypress_runner.js:1807:23)
    at Promise._settlePromiseFromHandler (http://localhost:5617/__cypress/runner/cypress_runner.js:1519:31)
    at Promise._settlePromise (http://localhost:5617/__cypress/runner/cypress_runner.js:1576:18)
    at Promise._settlePromise0 (http://localhost:5617/__cypress/runner/cypress_runner.js:1621:10)
    at Promise._settlePromises (http://localhost:5617/__cypress/runner/cypress_runner.js:1701:18)
    at Promise._fulfill (http://localhost:5617/__cypress/runner/cypress_runner.js:1645:18)
    at <unknown> (http://localhost:5617/__cypress/runner/cypress_runner.js:5450:46)
From Your Spec Code:
    at Context.eval (webpack:///./e2e/investigations/threat_intelligence/indicators.cy.ts:303:9)

First failure: kibana-on-merge - main

@kibanamachine kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Oct 10, 2024
@botelastic botelastic bot added the needs-team Issues missing a team label label Oct 10, 2024
@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine kibanamachine added the Team:Threat Hunting:Investigations Security Solution Investigations Team label Oct 10, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-threat-hunting-investigations (Team:Threat Hunting:Investigations)

@botelastic botelastic bot removed the needs-team Issues missing a team label label Oct 10, 2024
@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

1 similar comment
@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@jbudz
Copy link
Member

jbudz commented Oct 10, 2024

#192789

@jbudz
Copy link
Member

jbudz commented Oct 10, 2024

/skip

@kibanamachine
Copy link
Contributor Author

Skipped

main: 35bbb49

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

1 similar comment
@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

1 similar comment
@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@PhilippeOberti
Copy link
Contributor

PhilippeOberti commented Oct 10, 2024

#192789

@jbudz I'm curious about what this means? Are all the failures here due do this PR merged 4 hours ago? If yes, why don't we revert that PR that broke the tests instead of skipping the tests?

/skip

Also, why did we skip the Single indicator describe, while all the errors reported on all the buildkite runs here are on the Multiple indicators describe? We could skip only the one test (it seems that only the should handle all search actions is failing, no need to skip the whole describe?)

@jbudz
Copy link
Member

jbudz commented Oct 10, 2024

@jbudz I'm curious about what this means? Are all the failures here due do this PR merged 4 hours ago? If yes, why don't we revert that PR that broke the tests instead of skipping the tests?

Apologies! Ideally CI failures can be directly attributed to a commit, and reverted after. That was the case here and I could have handled it better.

What happened at the time: There were two other unrelated failures that could not be attributed to commits, and a timeout after 5 seconds here and I defaulted to skipping. The longer CI is red the more pull requests get blocked, so we prefer fixes we know will work and following up after. Following up involves blocking a release if one is in progress, and we believe there's a regression.

The linked PR came from text matching an older fix from one of the other issues, #184581 and thought it might be helpful to share.

Also, why did we skip the Single indicator describe, while all the errors reported on all the buildkite runs here are on the Multiple indicators describe? We could skip only the one test (it seems that only the should handle all search actions is failing, no need to skip the whole describe?)

We've tried this, but unfortunately many functional tests end up depending on previous steps. We're not familiar with the test, and the feedback loop is long.

@PhilippeOberti
Copy link
Contributor

@jbudz thanks for the explanation above, it makes sense! 😄

I have a PR to re-enable the test with the fix, hopefully my team can review and approve soon.

Thanks again!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocker failed-test A test failure on a tracked branch, potentially flaky-test skipped-test Team:Threat Hunting:Investigations Security Solution Investigations Team v9.0.0
Projects
None yet
Development

No branches or pull requests

4 participants