[Canvas] Fix flaky test failures when navigating to Canvas app #164376

cqliu1 · 2023-08-21T21:15:02Z

Summary

Closes #163303.
Closes #163368.
Closes #163903.

This fixes a rarely occurring flaky test failure while navigating to the Canvas app where the currentURL check fails. The cause is navigateToApp goes to app/canvas#/ which includes a hash get redirected by the Canvas app to app/canvas/, and fails when app/canvas/ doesn't match app/canvas#/. I've chosen to disable this check when navigating to Canvas to avoid this flakiness because I couldn't remove the hash from the URL while using navigateToApp or navigateToURL.

Instead of checking the URLs match, I opted to check that the workpad listing page loaded correctly.

Checklist

Delete any items that are not applicable to this PR.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
Any UI touched in this PR is usable by keyboard only (learn more about keyboard accessibility)
Any UI touched in this PR does not create any new axe failures (run axe in browser: FF, Chrome)
If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
This renders correctly on smaller devices using a responsive layout. (You can test this in your browser)
This was checked for cross-browser compatibility

Risk Matrix

Delete this section if it is not applicable to this PR.

Before closing this PR, invite QA, stakeholders, and other developers to identify risks that should be tested prior to the change/feature release.

When forming the risk matrix, consider some of the following examples and how they may potentially impact the change:

Risk	Probability	Severity	Mitigation/Notes
Multiple Spaces—unexpected behavior in non-default Kibana Space.	Low	High	Integration tests will verify that all features are still supported in non-default Kibana Space and when user switches between spaces.
Multiple nodes—Elasticsearch polling might have race conditions when multiple Kibana nodes are polling for the same tasks.	High	Low	Tasks are idempotent, so executing them multiple times will not result in logical error, but will degrade performance. To test for this case we add plenty of unit tests around this logic and document manual testing procedure.
Code should gracefully handle cases when feature X or plugin Y are disabled.	Medium	High	Unit tests will verify that any feature flag or plugin combination still results in our service operational.
See more potential risk examples

For maintainers

This was checked for breaking API changes and was labeled appropriately

cqliu1 · 2023-08-21T21:16:00Z

Flaky test runner x 100: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2942

elasticmachine · 2023-08-22T15:00:50Z

Pinging @elastic/kibana-presentation (Team:Presentation)

cqliu1 · 2023-08-22T17:11:31Z

Updated flaky test runner x 100 https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2953

ThomThomson

Test + page object changes LGTM!

ThomThomson · 2023-08-22T17:11:35Z

test/functional/page_objects/common_page.ts

+    // See https://github.com/elastic/kibana/pull/164376
+    if (appName === 'canvas' && !path) {
+      throw new Error(
+        'This causes flaky test failures. Use Canvas page object goToListingPage instead'


sebelga

LGTM!

…igate-to-canvas-flakiness

…/cqliu1/kibana into fix/navigate-to-canvas-flakiness

…igate-to-canvas-flakiness

…/cqliu1/kibana into fix/navigate-to-canvas-flakiness

cqliu1 · 2023-09-19T15:39:42Z

Flaky test runner x100: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3186

kibana-ci · 2023-09-19T16:01:25Z

💚 Build Succeeded

Buildkite Build
Commit: 6a92a42

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`canvas`	1011.6KB	1011.6KB	+34.0B

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Contributes to: elastic#194817 Why I assigned them: Assigned advanced_settings due to elastic#175255 Assigned dashboard_controls due to elastic#190797 Assigned dashboard_links due to elastic#174772 Assigned dashboard_panel_options due to elastic#178596 Assigned grok_debugger due to https://github.com/elastic/kibana/blob/main/x-pack/plugins/grokdebugger/kibana.jsonc#L4 Assigned helpers due to elastic#164341 (call site) Assigned home due to elastic#103192 Assigned index_lifecycle_management due to elastic#116207 Assigned ingest_node_pipelines due to elastic#113783 Assigned kibana_overview due to https://github.com/elastic/kibana/blob/f00ac7a8a21463e6bb4a2784c3a3884f36c62900/x-pack/plugins/grokdebugger/kibana.jsonc#L4 Assigned management due to elastic#165705 Assigned painless_lab due to https://github.com/elastic/kibana/blob/main/x-pack/plugins/painless_lab/kibana.jsonc#L4 Assigned search_profiler due to elastic#195343 Assigned uptime due to https://github.com/elastic/kibana/blob/main/x-pack/plugins/observability_solution/uptime/kibana.jsonc#L4 Assigned lens due to elastic#175893 Assigned ml_anomaly_detection due to elastic#162126 Assigned canvas due to elastic#164376 Assigned cc replication due to elastic#149069 Assigned enterpise search due to elastic#79359 Assigned graph to due elastic#190797 Assigned license_management due to https://github.com/elastic/kibana/blob/main/x-pack/plugins/license_management/kibana.jsonc#L4 Assigned maps due to elastic#155161 Assigned observability due to https://github.com/elastic/kibana/blob/main/x-pack/plugins/observability_solution/observability/kibana.jsonc#L4 Assigned remote clusters due to elastic#96989 Assigned reporting due to elastic#121435 Assigned rollup_jobs due to https://github.com/elastic/kibana/blob/d57bc9a5d7d64f86b550eff7997605a3090aee9a/x-pack/plugins/rollup/kibana.jsonc#L4 Assigned watcher due to elastic#119717

Remove current URL check when navigating to Canvas

70b2346

cqliu1 added 2 commits August 21, 2023 15:00

Fix canvas page objects

270434b

Remove exclusive suite

cdb4a6e

cqliu1 marked this pull request as ready for review August 22, 2023 15:00

cqliu1 requested review from a team as code owners August 22, 2023 15:00

ThomThomson self-requested a review August 22, 2023 15:46

Throw when passing canvas to navigateToApp

053fd91

cqliu1 requested a review from a team as a code owner August 22, 2023 17:09

Fix canvas page object

aa355d6

ThomThomson approved these changes Aug 22, 2023

View reviewed changes

Fix failing tests

ae64c2a

sebelga approved these changes Aug 23, 2023

View reviewed changes

cqliu1 added 7 commits August 23, 2023 11:39

Fix tests

47823dc

Fix ts errors

4cc34e0

Merge branch 'main' of https://github.com/elastic/kibana into fix/nav…

37b931e

…igate-to-canvas-flakiness

Fix canvas expression test

93e8065

Fix canvas filters tests

679f2d3

Fix page object

206bf46

Fix missing page object

f2c78b2

cqliu1 added 12 commits September 1, 2023 12:51

Merge branch 'main' into fix/navigate-to-canvas-flakiness

bf7edbb

Fix page object

85ab931

Merge branch 'fix/navigate-to-canvas-flakiness' of https://github.com…

647fe9b

…/cqliu1/kibana into fix/navigate-to-canvas-flakiness

Merge branch 'main' of https://github.com/elastic/kibana into fix/nav…

b38518e

…igate-to-canvas-flakiness

Merge branch 'main' into fix/navigate-to-canvas-flakiness

162eff9

Merge branch 'main' of https://github.com/elastic/kibana into fix/nav…

ccc0074

…igate-to-canvas-flakiness

Merge branch 'main' of https://github.com/elastic/kibana into fix/nav…

5ae931e

…igate-to-canvas-flakiness

Merge branch 'fix/navigate-to-canvas-flakiness' of https://github.com…

86cbdd8

…/cqliu1/kibana into fix/navigate-to-canvas-flakiness

Filter workpads in loadFirstWorkpad

21b910a

Fix save object resolve test

1e7dc0d

Remove exclusive suite

56e8be9

Fix typo

6a92a42

cqliu1 merged commit 94587c9 into elastic:main Sep 19, 2023
18 checks passed

kibanamachine added the backport:skip This commit does not require backporting label Sep 19, 2023

cqliu1 deleted the fix/navigate-to-canvas-flakiness branch September 19, 2023 20:10

cqliu1 mentioned this pull request Oct 2, 2023

Failing test: Chrome X-Pack UI Functional Tests.x-pack/test/functional/apps/canvas/custom_elements·ts - Canvas Canvas app custom elements "before all" hook for "creates a custom element from an element when prompted" #162820

Closed

wayneseymour mentioned this pull request Oct 11, 2024

[FTR][Ownership] Assign ally/grps 1,2 & 3 #195964

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Canvas] Fix flaky test failures when navigating to Canvas app #164376

[Canvas] Fix flaky test failures when navigating to Canvas app #164376

cqliu1 commented Aug 21, 2023 •

edited

Loading

cqliu1 commented Aug 21, 2023 •

edited

Loading

elasticmachine commented Aug 22, 2023

cqliu1 commented Aug 22, 2023 •

edited

Loading

ThomThomson left a comment

ThomThomson Aug 22, 2023

sebelga left a comment

cqliu1 commented Sep 19, 2023

kibana-ci commented Sep 19, 2023

[Canvas] Fix flaky test failures when navigating to Canvas app #164376

[Canvas] Fix flaky test failures when navigating to Canvas app #164376

Conversation

cqliu1 commented Aug 21, 2023 • edited Loading

Summary

Checklist

Risk Matrix

For maintainers

cqliu1 commented Aug 21, 2023 • edited Loading

elasticmachine commented Aug 22, 2023

cqliu1 commented Aug 22, 2023 • edited Loading

ThomThomson left a comment

Choose a reason for hiding this comment

ThomThomson Aug 22, 2023

Choose a reason for hiding this comment

sebelga left a comment

Choose a reason for hiding this comment

cqliu1 commented Sep 19, 2023

kibana-ci commented Sep 19, 2023

💚 Build Succeeded

Metrics [docs]

Async chunks

History

cqliu1 commented Aug 21, 2023 •

edited

Loading

cqliu1 commented Aug 21, 2023 •

edited

Loading

cqliu1 commented Aug 22, 2023 •

edited

Loading