Skip to content
This repository has been archived by the owner on Sep 11, 2024. It is now read-only.

End to End Tests

End to End Tests #7934

Triggered via workflow run March 8, 2024 11:38
@t3chguyt3chguy
completed ddbc643
Status Success
Total duration 19m 53s
Artifacts 9

end-to-end-tests.yaml

on: workflow_run
Prepare
2s
Prepare
Matrix: tests
Report results
34s
Report results
Fit to window
Zoom out
Zoom in

Annotations

10 errors, 7 warnings, and 1 notice
[Legacy Crypto] › crypto/verification.spec.ts:76:9 › Device verification › Verify device with SAS during login: matrix-react-sdk/playwright/e2e/crypto/utils.ts#L118
1) [Legacy Crypto] › crypto/verification.spec.ts:76:9 › Device verification › Verify device with SAS during login Error: Timed out 5000ms waiting for expect(locator).toBeVisible() Locator: locator('.mx_Dialog').getByRole('button', { name: 'Restore from Backup' }) Expected: visible Received: hidden Call log: - expect.toBeVisible with timeout 5000ms - waiting for locator('.mx_Dialog').getByRole('button', { name: 'Restore from Backup' }) at crypto/utils.ts:118 116 | await page.getByRole("button", { name: "User menu" }).click(); 117 | await page.locator(".mx_UserMenu_contextMenu").getByRole("menuitem", { name: "Security & Privacy" }).click(); > 118 | await expect(page.locator(".mx_Dialog").getByRole("button", { name: "Restore from Backup" })).toBeVisible(); | ^ 119 | 120 | // expand the advanced section to see the active version in the reports 121 | await page.locator(".mx_SecureBackupPanel_advanced").locator("..").click(); at checkDeviceIsConnectedKeyBackup (/home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/crypto/utils.ts:118:99) at /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/crypto/verification.spec.ts:99:9
[Legacy Crypto] › room/room.spec.ts:64:9 › Room Directory › should memorize the timeline position when switch Room A -> Room B -> Room A: matrix-react-sdk/playwright/e2e/room/room.spec.ts#L100
2) [Legacy Crypto] › room/room.spec.ts:64:9 › Room Directory › should memorize the timeline position when switch Room A -> Room B -> Room A Error: Timed out 5000ms waiting for expect(locator).toBeInViewport() Locator: locator('.mx_MTextBody .mx_EventTile_body').getByText('test0') Expected: in viewport Received: outside viewport Call log: - expect.toBeInViewport with timeout 5000ms - waiting for locator('.mx_MTextBody .mx_EventTile_body').getByText('test0') 98 | // The timeline should display the first message 99 | // The previous position before switching to Room B should be remembered > 100 | await expect(page.locator(".mx_MTextBody .mx_EventTile_body").getByText("test0")).toBeInViewport(); | ^ 101 | }); 102 | }); 103 | at /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/room/room.spec.ts:100:91
[Legacy Crypto] › timeline/timeline.spec.ts:910:13 › Timeline › message sending › should display a reply chain: matrix-react-sdk/playwright/e2e/timeline/timeline.spec.ts#L1017
3) [Legacy Crypto] › timeline/timeline.spec.ts:910:13 › Timeline › message sending › should display a reply chain Error: Screenshot comparison failed: 103 pixels (ratio 0.01 of all image pixels) are different. Expected: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/snapshots/timeline/timeline.spec.ts/event-tile-reply-chains-bubble-layout-linux.png Received: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/test-results/timeline-timeline-Timeline-message-sending-should-display-a-reply-chain-Legacy-Crypto/event-tile-reply-chains-bubble-layout-actual.png Diff: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/test-results/timeline-timeline-Timeline-message-sending-should-display-a-reply-chain-Legacy-Crypto/event-tile-reply-chains-bubble-layout-diff.png Call log: - page._expectScreenshot with timeout 5000ms - verifying given screenshot expectation - waiting for locator('.mx_EventTile_last') - locator resolved to <li tabindex="-1" aria-live="off" data-self="true" a…>…</li> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - 103 pixels (ratio 0.01 of all image pixels) are different. - waiting 100ms before taking screenshot - waiting for locator('.mx_EventTile_last') - locator resolved to <li tabindex="-1" aria-live="off" data-self="true" a…>…</li> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - captured a stable screenshot - 103 pixels (ratio 0.01 of all image pixels) are different. 1015 | 1016 | // Take a snapshot on bubble layout > 1017 | await expect(page.locator(".mx_EventTile_last")).toMatchScreenshot( | ^ 1018 | "event-tile-reply-chains-bubble-layout.png", 1019 | screenshotOptions, 1020 | ); at /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/timeline/timeline.spec.ts:1017:62
[Legacy Crypto] › user-onboarding/user-onboarding-new.spec.ts:43:9 › User Onboarding (new user) › app download dialog: matrix-react-sdk/playwright/e2e/user-onboarding/user-onboarding-new.spec.ts#L48
4) [Legacy Crypto] › user-onboarding/user-onboarding-new.spec.ts:43:9 › User Onboarding (new user) › app download dialog Error: Screenshot comparison failed: 8 pixels (ratio 0.01 of all image pixels) are different. Expected: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/snapshots/user-onboarding/user-onboarding-new.spec.ts/User-Onboarding-new-user-app-download-dialog-1-linux.png Received: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/test-results/user-onboarding-user-onboarding-new-User-Onboarding-new-user-app-download-dialog-Legacy-Crypto/User-Onboarding-new-user-app-download-dialog-1-actual.png Diff: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/test-results/user-onboarding-user-onboarding-new-User-Onboarding-new-user-app-download-dialog-Legacy-Crypto/User-Onboarding-new-user-app-download-dialog-1-diff.png Call log: - page._expectScreenshot with timeout 5000ms - verifying given screenshot expectation - waiting for locator('.mx_Dialog') - locator resolved to <div class="mx_Dialog">…</div> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - 8 pixels (ratio 0.01 of all image pixels) are different. - waiting 100ms before taking screenshot - waiting for locator('.mx_Dialog') - locator resolved to <div class="mx_Dialog">…</div> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - captured a stable screenshot - 8 pixels (ratio 0.01 of all image pixels) are different. 46 | page.getByRole("dialog").getByRole("heading", { level: 1, name: "Download Element" }), 47 | ).toBeVisible(); > 48 | await expect(page.locator(".mx_Dialog")).toMatchScreenshot(); | ^ 49 | }); 50 | 51 | test("using find friends action should increase progress", async ({ page, homeserver }) => { at /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/user-onboarding/user-onboarding-new.spec.ts:48:50
[Rust Crypto] › audio-player/audio-player.spec.ts:146:9 › Audio player › should be correctly rendered - light theme with monospace font: matrix-react-sdk/playwright/e2e/audio-player/audio-player.spec.ts#L81
5) [Rust Crypto] › audio-player/audio-player.spec.ts:146:9 › Audio player › should be correctly rendered - light theme with monospace font Error: expect(received).toBeGreaterThan(expected) Expected: > 0 Received: 0 79 | 80 | // Check the status of the seek bar > 81 | expect(await page.locator(".mx_AudioPlayer_seek input[type='range']").count()).toBeGreaterThan(0); | ^ 82 | 83 | // Enable IRC layout 84 | await app.settings.setValue("layout", null, SettingLevel.DEVICE, Layout.IRC); at takeSnapshots (/home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/audio-player/audio-player.spec.ts:81:88) at /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/audio-player/audio-player.spec.ts:149:9
[Rust Crypto] › audio-player/audio-player.spec.ts:152:9 › Audio player › should be correctly rendered - high contrast theme: matrix-react-sdk/playwright/e2e/audio-player/audio-player.spec.ts#L103
6) [Rust Crypto] › audio-player/audio-player.spec.ts:152:9 › Audio player › should be correctly rendered - high contrast theme Error: Screenshot comparison failed: 154 pixels (ratio 0.01 of all image pixels) are different. Expected: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/snapshots/audio-player/audio-player.spec.ts/Selected-EventTile-of-audio-player--high-contrast--irc-layout-linux.png Received: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/test-results/audio-player-audio-player-Audio-player-should-be-correctly-rendered---high-contrast-theme-Rust-Crypto/Selected-EventTile-of-audio-player--high-contrast--irc-layout-actual.png Diff: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/test-results/audio-player-audio-player-Audio-player-should-be-correctly-rendered---high-contrast-theme-Rust-Crypto/Selected-EventTile-of-audio-player--high-contrast--irc-layout-diff.png Call log: - page._expectScreenshot with timeout 5000ms - verifying given screenshot expectation - waiting for locator('.mx_EventTile_last') - locator resolved to <li tabindex="-1" aria-live="off" data-self="true" d…>…</li> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - 154 pixels (ratio 0.01 of all image pixels) are different. - waiting 100ms before taking screenshot - waiting for locator('.mx_EventTile_last') - locator resolved to <li tabindex="-1" aria-live="off" data-self="true" d…>…</li> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - captured a stable screenshot - 154 pixels (ratio 0.01 of all image pixels) are different. 101 | 102 | // Take a snapshot of mx_EventTile_last on IRC layout > 103 | await expect(page.locator(".mx_EventTile_last")).toMatchScreenshot( | ^ 104 | `${detail.replaceAll(" ", "-")}-irc-layout.png`, 105 | screenshotOptions, 106 | ); at takeSnapshots (/home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/audio-player/audio-player.spec.ts:103:58) at /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/audio-player/audio-player.spec.ts:165:9
[Rust Crypto] › oidc/oidc-native.spec.ts:28:9 › OIDC Native › can register the oauth2 client and an account: matrix-react-sdk/playwright/e2e/oidc/oidc-native.spec.ts#L1
7) [Rust Crypto] › oidc/oidc-native.spec.ts:28:9 › OIDC Native › can register the oauth2 client and an account Test timeout of 90000ms exceeded.
[Rust Crypto] › oidc/oidc-native.spec.ts:28:9 › OIDC Native › can register the oauth2 client and an account: matrix-react-sdk/playwright/e2e/oidc/oidc-native.spec.ts#L30
7) [Rust Crypto] › oidc/oidc-native.spec.ts:28:9 › OIDC Native › can register the oauth2 client and an account Error: page.waitForRequest: Test timeout of 90000ms exceeded. 28 | test("can register the oauth2 client and an account", async ({ context, page, homeserver, mailhog, app, mas }) => { 29 | const tokenUri = `http://localhost:${mas.port}/oauth2/token`; > 30 | const tokenApiPromise = page.waitForRequest( | ^ 31 | (request) => request.url() === tokenUri && request.postDataJSON()["grant_type"] === "authorization_code", 32 | ); 33 | at /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/oidc/oidc-native.spec.ts:30:38
[Rust Crypto] › read-receipts/reactions.spec.ts:24:17 › Read receipts › reactions › in the main timeline › Receiving a reaction to a message does not make a room unread: matrix-react-sdk/playwright/e2e/read-receipts/reactions.spec.ts#L1
8) [Rust Crypto] › read-receipts/reactions.spec.ts:24:17 › Read receipts › reactions › in the main timeline › Receiving a reaction to a message does not make a room unread Test finished within timeout of 30000ms, but tearing down "page" ran out of time. Please allow more time for the test, since teardown is attributed towards the test timeout budget.
[Rust Crypto] › room/room.spec.ts:64:9 › Room Directory › should memorize the timeline position when switch Room A -> Room B -> Room A: matrix-react-sdk/playwright/e2e/room/room.spec.ts#L100
9) [Rust Crypto] › room/room.spec.ts:64:9 › Room Directory › should memorize the timeline position when switch Room A -> Room B -> Room A Error: Timed out 5000ms waiting for expect(locator).toBeInViewport() Locator: locator('.mx_MTextBody .mx_EventTile_body').getByText('test0') Expected: in viewport Received: outside viewport Call log: - expect.toBeInViewport with timeout 5000ms - waiting for locator('.mx_MTextBody .mx_EventTile_body').getByText('test0') 98 | // The timeline should display the first message 99 | // The previous position before switching to Room B should be remembered > 100 | await expect(page.locator(".mx_MTextBody .mx_EventTile_body").getByText("test0")).toBeInViewport(); | ^ 101 | }); 102 | }); 103 | at /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/room/room.spec.ts:100:91
Prepare
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: Sibz/github-status-action@071b5370da85afbb16637d6eed8524a06bc2053e. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Report results
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: Sibz/github-status-action@071b5370da85afbb16637d6eed8524a06bc2053e, bobheadxi/deployments@88ce5600046c82542f8246ac287d0a53c461bca3, actions/github-script@v6, nwtgck/actions-netlify@7a92f00dde8c92a5a9e8385ec2919775f7647352. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Slow Test: [Rust Crypto] › read-receipts/redactions.spec.ts#L1
[Rust Crypto] › read-receipts/redactions.spec.ts took 5.9m
Slow Test: [Legacy Crypto] › read-receipts/redactions.spec.ts#L1
[Legacy Crypto] › read-receipts/redactions.spec.ts took 5.4m
Slow Test: [Rust Crypto] › read-receipts/new-messages.spec.ts#L1
[Rust Crypto] › read-receipts/new-messages.spec.ts took 3.8m
Slow Test: [Rust Crypto] › read-receipts/high-level.spec.ts#L1
[Rust Crypto] › read-receipts/high-level.spec.ts took 3.6m
Slow Test: [Legacy Crypto] › read-receipts/new-messages.spec.ts#L1
[Legacy Crypto] › read-receipts/new-messages.spec.ts took 3.4m
🎭 Playwright Run Summary
9 flaky [Legacy Crypto] › crypto/verification.spec.ts:76:9 › Device verification › Verify device with SAS during login [Legacy Crypto] › room/room.spec.ts:64:9 › Room Directory › should memorize the timeline position when switch Room A -> Room B -> Room A [Legacy Crypto] › timeline/timeline.spec.ts:910:13 › Timeline › message sending › should display a reply chain [Legacy Crypto] › user-onboarding/user-onboarding-new.spec.ts:43:9 › User Onboarding (new user) › app download dialog [Rust Crypto] › audio-player/audio-player.spec.ts:146:9 › Audio player › should be correctly rendered - light theme with monospace font [Rust Crypto] › audio-player/audio-player.spec.ts:152:9 › Audio player › should be correctly rendered - high contrast theme [Rust Crypto] › oidc/oidc-native.spec.ts:28:9 › OIDC Native › can register the oauth2 client and an account [Rust Crypto] › read-receipts/reactions.spec.ts:24:17 › Read receipts › reactions › in the main timeline › Receiving a reaction to a message does not make a room unread [Rust Crypto] › room/room.spec.ts:64:9 › Room Directory › should memorize the timeline position when switch Room A -> Room B -> Room A 121 skipped 654 passed (18.3m)

Artifacts

Produced during runtime
Name Size
all-blob-reports-1 Expired
11.2 MB
all-blob-reports-2 Expired
615 KB
all-blob-reports-3 Expired
6.08 MB
all-blob-reports-4 Expired
7.43 MB
all-blob-reports-5 Expired
13.1 MB
all-blob-reports-6 Expired
589 KB
all-blob-reports-7 Expired
8.52 MB
all-blob-reports-8 Expired
1.11 MB
html-report--attempt-1 Expired
43.7 MB