[ci] Move cypress burn tests out of base pull request pipeline

[jbudz]

Currently operations does not have a good mechanism for communicating soft failures, leading to confusion on pull requests. We're tracking support for soft failures at #169244.

For example, in #169309, it doesn't makes sense for the author to be responsible for fixing flakiness here, so we need to make it clear it can be ignored or find a way to skip this step in similar situations.

This moves the cypress burn out of the base pull request pipeline, behind the ci:all-cypress-suites and ci:cypress-burn labels. I'm open to all improvements here, not sure this quite finds the right balance between stability and noise.

[elasticmachine]

Pinging @elastic/kibana-operations (Team:Operations)

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[kibana-ci]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: 902e6c1

Failed CI Steps

• Serverless Osquery Cypress Tests #1

Test Failures

• [job] [logs] Serverless Osquery Cypress Tests #1 / ALL - Add Integration Add and upgrade integration "after each" hook for "should add the old integration and be able to upgrade it" "after each" hook for "should add the old integration and be able to upgrade it"
• [job] [logs] Serverless Osquery Cypress Tests #1 / ALL - Add Integration Add and upgrade integration should add the old integration and be able to upgrade it should add the old integration and be able to upgrade it

Metrics [docs]

✅ unchanged

History

• 💚 Build #169275 succeeded e3f6fb9

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream