#10793 Fix playwright tests for Windows and MacOS #10826

ndoschek · 2022-03-02T08:39:09Z

What it does

Fix workspace path encoding for Windows
Stabilize workspace path fetching
Fix explorer review title button selectors
Introduce OSUtil helper functions
Fix shortcuts for MacOS
Improve opening of TheiaViews (speeds up tests by approx. 10s)

Contributed on behalf of STMicroelectronics

Signed-off-by: Nina Doschek ndoschek@eclipsesource.com

Note: I ran tests multiple times on Linux, MacOS and Windows and can confirm that tests are passing (different execution times due to different system specs).

How to test

Run playwright tests:

yarn
yarn browser build
yarn test:playwright
Confirm tests pass on different OS

Review checklist

As an author, I have thoroughly tested my changes and carefully followed the review guidelines

Reminder for reviewers

As a reviewer, I agree to behave in accordance with the review guidelines

JonasHelming · 2022-03-08T16:02:54Z

@colin-grant-work : As far as I remember you had a mac, right? Would you mind evalluating the fixes on the Playwright tests on your machine?

colin-grant-work · 2022-03-11T15:50:02Z

I've still got quite a number of failures running this code on MacOS:

[playwright]   Slow test file: ../../tests/theia-quick-command.test.ts (4m)
[playwright]   Slow test file: ../../tests/theia-explorer-view.test.ts (2m)
[playwright]   Slow test file: ../../tests/theia-problems-view.test.ts (2m)
[playwright]   Slow test file: ../../tests/theia-preference-view.test.ts (2m)
[playwright]   Consider splitting slow test files to speed up parallel execution
[playwright] 
[playwright]   9 failed
[playwright]     ../../tests/theia-explorer-view.test.ts:46:9 › Theia Explorer View › should be possible to close and reopen it 
[playwright]     ../../tests/theia-explorer-view.test.ts:56:9 › Theia Explorer View › should show one folder named "sampleFolder" and one file named "sample.txt" 
[playwright]     ../../tests/theia-preference-view.test.ts:30:9 › Preference View › should be visible and active after being opened 
[playwright]     ../../tests/theia-preference-view.test.ts:37:9 › Preference View › should be able to read, set, and reset String preferences 
[playwright]     ../../tests/theia-problems-view.test.ts:30:9 › Theia Problems View › should be visible and active after being opened 
[playwright]     ../../tests/theia-problems-view.test.ts:37:9 › Theia Problems View › should be opened at the bottom and have the title 'Problems' 
[playwright]     ../../tests/theia-quick-command.test.ts:34:9 › Theia Quick Command › should show quick command palette 
[playwright]     ../../tests/theia-quick-command.test.ts:39:9 › Theia Quick Command › should trigger 'About' command after typing 
[playwright]     ../../tests/theia-quick-command.test.ts:53:9 › Theia Quick Command › should trigger 'Toggle Explorer View' command after typing 
[playwright]   10 skipped
[playwright]   29 passed (11m)

ndoschek · 2022-03-11T16:08:27Z

Thanks for your feedback @colin-grant-work!
Could you please post the the upper part of the output log with the error desriptions as well (and if any, the failure screenshots from examples/playwright/test-results). That would help a lot to investigate, thanks!

colin-grant-work · 2022-03-11T18:03:20Z

@ndoschek, here is the full failure log:

[playwright] 
[playwright] 
[playwright]   1) ../../tests/theia-explorer-view.test.ts:46:9 › Theia Explorer View › should be possible to close and reopen it 
[playwright] 
[playwright]     Timeout of 60000ms exceeded.
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget .monaco-inputbox .input" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]       48 |         expect(await explorer.isTabVisible()).toBe(false);
[playwright]       49 |
[playwright]     > 50 |         explorer = await app.openView(TheiaExplorerView);
[playwright]          |                    ^
[playwright]       51 |         expect(await explorer.isTabVisible()).toBe(true);
[playwright]       52 |         expect(await explorer.isDisplayed()).toBe(true);
[playwright]       53 |         expect(await explorer.isActive()).toBe(true);
[playwright] 
[playwright]         at TheiaQuickCommandPalette.type (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:61:39)
[playwright]         at TheiaExplorerView.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-view.ts:53:9)
[playwright]         at TheiaApp.openView (/Users/cjg/code/open/theia/examples/playwright/src/theia-app.ts:83:9)
[playwright]         at /Users/cjg/code/open/theia/examples/playwright/tests/theia-explorer-view.test.ts:50:20
[playwright] 
[playwright]     attachment #1: screenshot (image/png) ----------------------------------------------------------
[playwright]     test-results/theia-explorer-view-Theia-Explorer-View-should-be-possible-to-close-and-reopen-it/test-failed-1.png
[playwright]     ------------------------------------------------------------------------------------------------
[playwright] 
[playwright]     Retry #1 ---------------------------------------------------------------------------------------
[playwright] 
[playwright]     Timeout of 60000ms exceeded.
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget .monaco-inputbox .input" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]       48 |         expect(await explorer.isTabVisible()).toBe(false);
[playwright]       49 |
[playwright]     > 50 |         explorer = await app.openView(TheiaExplorerView);
[playwright]          |                    ^
[playwright]       51 |         expect(await explorer.isTabVisible()).toBe(true);
[playwright]       52 |         expect(await explorer.isDisplayed()).toBe(true);
[playwright]       53 |         expect(await explorer.isActive()).toBe(true);
[playwright] 
[playwright]         at TheiaQuickCommandPalette.type (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:61:39)
[playwright]         at TheiaExplorerView.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-view.ts:53:9)
[playwright]         at TheiaApp.openView (/Users/cjg/code/open/theia/examples/playwright/src/theia-app.ts:83:9)
[playwright]         at /Users/cjg/code/open/theia/examples/playwright/tests/theia-explorer-view.test.ts:50:20
[playwright] 
[playwright]     attachment #1: screenshot (image/png) ----------------------------------------------------------
[playwright]     test-results/theia-explorer-view-Theia-Explorer-View-should-be-possible-to-close-and-reopen-it-retry1/test-failed-1.png
[playwright]     ------------------------------------------------------------------------------------------------
[playwright] 
[playwright]   2) ../../tests/theia-explorer-view.test.ts:56:9 › Theia Explorer View › should show one folder named "sampleFolder" and one file named "sample.txt" 
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]         at TheiaQuickCommandPalette.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:27:25)
[playwright] 
[playwright]     Retry #1 ---------------------------------------------------------------------------------------
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]         at TheiaQuickCommandPalette.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:27:25)
[playwright] 
[playwright]   3) ../../tests/theia-preference-view.test.ts:30:9 › Preference View › should be visible and active after being opened 
[playwright] 
[playwright]     Timeout of 60000ms exceeded.
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]       29 |
[playwright]       30 |     test('should be visible and active after being opened', async () => {
[playwright]     > 31 |         const preferenceView = await app.openPreferences(TheiaPreferenceView);
[playwright]          |                                ^
[playwright]       32 |         expect(await preferenceView.isTabVisible()).toBe(true);
[playwright]       33 |         expect(await preferenceView.isDisplayed()).toBe(true);
[playwright]       34 |         expect(await preferenceView.isActive()).toBe(true);
[playwright] 
[playwright]         at TheiaQuickCommandPalette.selectedCommand (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:69:41)
[playwright]         at TheiaQuickCommandPalette.triggerSingleCommand (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:49:35)
[playwright]         at TheiaQuickCommandPalette.trigger (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:41:13)
[playwright]         at TheiaPreferenceView.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-preference-view.ts:59:9)
[playwright]         at TheiaApp.openPreferences (/Users/cjg/code/open/theia/examples/playwright/src/theia-app.ts:73:9)
[playwright]         at /Users/cjg/code/open/theia/examples/playwright/tests/theia-preference-view.test.ts:31:32
[playwright] 
[playwright]     attachment #1: screenshot (image/png) ----------------------------------------------------------
[playwright]     test-results/theia-preference-view-Preference-View-should-be-visible-and-active-after-being-opened/test-failed-1.png
[playwright]     ------------------------------------------------------------------------------------------------
[playwright] 
[playwright]     Retry #1 ---------------------------------------------------------------------------------------
[playwright] 
[playwright]     Timeout of 60000ms exceeded.
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]       29 |
[playwright]       30 |     test('should be visible and active after being opened', async () => {
[playwright]     > 31 |         const preferenceView = await app.openPreferences(TheiaPreferenceView);
[playwright]          |                                ^
[playwright]       32 |         expect(await preferenceView.isTabVisible()).toBe(true);
[playwright]       33 |         expect(await preferenceView.isDisplayed()).toBe(true);
[playwright]       34 |         expect(await preferenceView.isActive()).toBe(true);
[playwright] 
[playwright]         at TheiaQuickCommandPalette.selectedCommand (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:69:41)
[playwright]         at TheiaQuickCommandPalette.triggerSingleCommand (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:49:35)
[playwright]         at TheiaQuickCommandPalette.trigger (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:41:13)
[playwright]         at TheiaPreferenceView.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-preference-view.ts:59:9)
[playwright]         at TheiaApp.openPreferences (/Users/cjg/code/open/theia/examples/playwright/src/theia-app.ts:73:9)
[playwright]         at /Users/cjg/code/open/theia/examples/playwright/tests/theia-preference-view.test.ts:31:32
[playwright] 
[playwright]     attachment #1: screenshot (image/png) ----------------------------------------------------------
[playwright]     test-results/theia-preference-view-Preference-View-should-be-visible-and-active-after-being-opened-retry1/test-failed-1.png
[playwright]     ------------------------------------------------------------------------------------------------
[playwright] 
[playwright]   4) ../../tests/theia-preference-view.test.ts:37:9 › Preference View › should be able to read, set, and reset String preferences 
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]         at TheiaQuickCommandPalette.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:27:25)
[playwright] 
[playwright]     Retry #1 ---------------------------------------------------------------------------------------
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]         at TheiaQuickCommandPalette.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:27:25)
[playwright] 
[playwright]   5) ../../tests/theia-problems-view.test.ts:30:9 › Theia Problems View › should be visible and active after being opened 
[playwright] 
[playwright]     Timeout of 60000ms exceeded.
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget .monaco-inputbox .input" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]       29 |
[playwright]       30 |     test('should be visible and active after being opened', async () => {
[playwright]     > 31 |         const problemsView = await app.openView(TheiaProblemsView);
[playwright]          |                              ^
[playwright]       32 |         expect(await problemsView.isTabVisible()).toBe(true);
[playwright]       33 |         expect(await problemsView.isDisplayed()).toBe(true);
[playwright]       34 |         expect(await problemsView.isActive()).toBe(true);
[playwright] 
[playwright]         at TheiaQuickCommandPalette.type (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:61:39)
[playwright]         at TheiaProblemsView.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-view.ts:53:9)
[playwright]         at TheiaApp.openView (/Users/cjg/code/open/theia/examples/playwright/src/theia-app.ts:83:9)
[playwright]         at /Users/cjg/code/open/theia/examples/playwright/tests/theia-problems-view.test.ts:31:30
[playwright] 
[playwright]     attachment #1: screenshot (image/png) ----------------------------------------------------------
[playwright]     test-results/theia-problems-view-Theia-Problems-View-should-be-visible-and-active-after-being-opened/test-failed-1.png
[playwright]     ------------------------------------------------------------------------------------------------
[playwright] 
[playwright]     Retry #1 ---------------------------------------------------------------------------------------
[playwright] 
[playwright]     Timeout of 60000ms exceeded.
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget .monaco-inputbox .input" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]       29 |
[playwright]       30 |     test('should be visible and active after being opened', async () => {
[playwright]     > 31 |         const problemsView = await app.openView(TheiaProblemsView);
[playwright]          |                              ^
[playwright]       32 |         expect(await problemsView.isTabVisible()).toBe(true);
[playwright]       33 |         expect(await problemsView.isDisplayed()).toBe(true);
[playwright]       34 |         expect(await problemsView.isActive()).toBe(true);
[playwright] 
[playwright]         at TheiaQuickCommandPalette.type (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:61:39)
[playwright]         at TheiaProblemsView.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-view.ts:53:9)
[playwright]         at TheiaApp.openView (/Users/cjg/code/open/theia/examples/playwright/src/theia-app.ts:83:9)
[playwright]         at /Users/cjg/code/open/theia/examples/playwright/tests/theia-problems-view.test.ts:31:30
[playwright] 
[playwright]     attachment #1: screenshot (image/png) ----------------------------------------------------------
[playwright]     test-results/theia-problems-view-Theia-Problems-View-should-be-visible-and-active-after-being-opened-retry1/test-failed-1.png
[playwright]     ------------------------------------------------------------------------------------------------
[playwright] 
[playwright]   6) ../../tests/theia-problems-view.test.ts:37:9 › Theia Problems View › should be opened at the bottom and have the title 'Problems' 
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]         at TheiaQuickCommandPalette.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:27:25)
[playwright] 
[playwright]     Retry #1 ---------------------------------------------------------------------------------------
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]         at TheiaQuickCommandPalette.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:27:25)
[playwright] 
[playwright]   7) ../../tests/theia-quick-command.test.ts:34:9 › Theia Quick Command › should show quick command palette 
[playwright] 
[playwright]     Timeout of 60000ms exceeded.
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]       33 |
[playwright]       34 |     test('should show quick command palette', async () => {
[playwright]     > 35 |         await quickCommand.open();
[playwright]          |         ^
[playwright]       36 |         expect(await quickCommand.isOpen()).toBe(true);
[playwright]       37 |     });
[playwright]       38 |
[playwright] 
[playwright]         at TheiaQuickCommandPalette.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:27:25)
[playwright]         at /Users/cjg/code/open/theia/examples/playwright/tests/theia-quick-command.test.ts:35:9
[playwright] 
[playwright]     attachment #1: screenshot (image/png) ----------------------------------------------------------
[playwright]     test-results/theia-quick-command-Theia-Quick-Command-should-show-quick-command-palette/test-failed-1.png
[playwright]     ------------------------------------------------------------------------------------------------
[playwright] 
[playwright]     Retry #1 ---------------------------------------------------------------------------------------
[playwright] 
[playwright]     Timeout of 60000ms exceeded.
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]       33 |
[playwright]       34 |     test('should show quick command palette', async () => {
[playwright]     > 35 |         await quickCommand.open();
[playwright]          |         ^
[playwright]       36 |         expect(await quickCommand.isOpen()).toBe(true);
[playwright]       37 |     });
[playwright]       38 |
[playwright] 
[playwright]         at TheiaQuickCommandPalette.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:27:25)
[playwright]         at /Users/cjg/code/open/theia/examples/playwright/tests/theia-quick-command.test.ts:35:9
[playwright] 
[playwright]     attachment #1: screenshot (image/png) ----------------------------------------------------------
[playwright]     test-results/theia-quick-command-Theia-Quick-Command-should-show-quick-command-palette-retry1/test-failed-1.png
[playwright]     ------------------------------------------------------------------------------------------------
[playwright] 
[playwright]   8) ../../tests/theia-quick-command.test.ts:39:9 › Theia Quick Command › should trigger 'About' command after typing 
[playwright] 
[playwright]     Timeout of 60000ms exceeded.
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget .monaco-inputbox .input" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]       38 |
[playwright]       39 |     test('should trigger \'About\' command after typing', async () => {
[playwright]     > 40 |         await quickCommand.type('About');
[playwright]          |         ^
[playwright]       41 |         await quickCommand.trigger('About');
[playwright]       42 |         expect(await quickCommand.isOpen()).toBe(false);
[playwright]       43 |         const aboutDialog = new TheiaAboutDialog(app);
[playwright] 
[playwright]         at TheiaQuickCommandPalette.type (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:61:39)
[playwright]         at /Users/cjg/code/open/theia/examples/playwright/tests/theia-quick-command.test.ts:40:9
[playwright] 
[playwright]     attachment #1: screenshot (image/png) ----------------------------------------------------------
[playwright]     test-results/theia-quick-command-Theia-Quick-Command-should-trigger-About-command-after-typing/test-failed-1.png
[playwright]     ------------------------------------------------------------------------------------------------
[playwright] 
[playwright]     Retry #1 ---------------------------------------------------------------------------------------
[playwright] 
[playwright]     Timeout of 60000ms exceeded.
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget .monaco-inputbox .input" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]       38 |
[playwright]       39 |     test('should trigger \'About\' command after typing', async () => {
[playwright]     > 40 |         await quickCommand.type('About');
[playwright]          |         ^
[playwright]       41 |         await quickCommand.trigger('About');
[playwright]       42 |         expect(await quickCommand.isOpen()).toBe(false);
[playwright]       43 |         const aboutDialog = new TheiaAboutDialog(app);
[playwright] 
[playwright]         at TheiaQuickCommandPalette.type (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:61:39)
[playwright]         at /Users/cjg/code/open/theia/examples/playwright/tests/theia-quick-command.test.ts:40:9
[playwright] 
[playwright]     attachment #1: screenshot (image/png) ----------------------------------------------------------
[playwright]     test-results/theia-quick-command-Theia-Quick-Command-should-trigger-About-command-after-typing-retry1/test-failed-1.png
[playwright]     ------------------------------------------------------------------------------------------------
[playwright] 
[playwright]   9) ../../tests/theia-quick-command.test.ts:53:9 › Theia Quick Command › should trigger 'Toggle Explorer View' command after typing 
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]         at TheiaQuickCommandPalette.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:27:25)
[playwright] 
[playwright]     Retry #1 ---------------------------------------------------------------------------------------
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]         at TheiaQuickCommandPalette.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:27:25)
[playwright] 
[playwright]   10) ../../tests/theia-explorer-view.test.ts:79:9 › Theia Explorer View › should provide file stat node by single path fragment "sample.txt" 
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]         at TheiaQuickCommandPalette.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:27:25)
[playwright] 
[playwright]     Retry #1 ---------------------------------------------------------------------------------------
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]         at TheiaQuickCommandPalette.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:27:25)
[playwright] 
[playwright]   11) ../../tests/theia-explorer-view.test.ts:86:9 › Theia Explorer View › should provide file stat nodes that can define whether they are collapsed or not and that can be expanded 
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]         at TheiaQuickCommandPalette.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:27:25)
[playwright] 
[playwright]     Retry #1 ---------------------------------------------------------------------------------------
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]         at TheiaQuickCommandPalette.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:27:25)
[playwright] 
[playwright]   12) ../../tests/theia-explorer-view.test.ts:97:9 › Theia Explorer View › should provide file stat node by path "sampleFolder/sampleFolder1/sampleFolder1-1/sampleFile1-1-1.txt" 
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]         at TheiaQuickCommandPalette.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:27:25)
[playwright] 
[playwright]     Retry #1 ---------------------------------------------------------------------------------------
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]         at TheiaQuickCommandPalette.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:27:25)
[playwright] 
[playwright]   13) ../../tests/theia-explorer-view.test.ts:103:9 › Theia Explorer View › should open context menu on "sample.txt" 
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]         at TheiaQuickCommandPalette.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:27:25)
[playwright] 
[playwright]     Retry #1 ---------------------------------------------------------------------------------------
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]         at TheiaQuickCommandPalette.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:27:25)
[playwright] 
[playwright]   14) ../../tests/theia-explorer-view.test.ts:117:9 › Theia Explorer View › should rename "sample.txt" 
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]         at TheiaQuickCommandPalette.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:27:25)
[playwright] 
[playwright]     Retry #1 ---------------------------------------------------------------------------------------
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]         at TheiaQuickCommandPalette.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:27:25)
[playwright] 
[playwright]   15) ../../tests/theia-preference-view.test.ts:52:9 › Preference View › should be able to read, set, and reset Boolean preferences 
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]         at TheiaQuickCommandPalette.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:27:25)
[playwright] 
[playwright]     Retry #1 ---------------------------------------------------------------------------------------
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]         at TheiaQuickCommandPalette.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:27:25)
[playwright] 
[playwright]   16) ../../tests/theia-preference-view.test.ts:67:9 › Preference View › should throw an error if we try to read, set, or reset a non-existing preference 
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]         at TheiaQuickCommandPalette.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:27:25)
[playwright] 
[playwright]     Retry #1 ---------------------------------------------------------------------------------------
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]         at TheiaQuickCommandPalette.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:27:25)
[playwright] 
[playwright]   17) ../../tests/theia-preference-view.test.ts:84:9 › Preference View › should throw an error if we try to read, or set a preference with the wrong type 
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]         at TheiaQuickCommandPalette.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:27:25)
[playwright] 
[playwright]     Retry #1 ---------------------------------------------------------------------------------------
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]         at TheiaQuickCommandPalette.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:27:25)
[playwright] 
[playwright]   18) ../../tests/theia-problems-view.test.ts:44:9 › Theia Problems View › should be closable ======
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]         at TheiaQuickCommandPalette.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:27:25)
[playwright] 
[playwright]     Retry #1 ---------------------------------------------------------------------------------------
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]         at TheiaQuickCommandPalette.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:27:25)
[playwright] 
[playwright]   19) ../../tests/theia-problems-view.test.ts:54:9 › Theia Problems View › should not throw an error if 'close' is called twice 
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]         at TheiaQuickCommandPalette.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:27:25)
[playwright] 
[playwright]     Retry #1 ---------------------------------------------------------------------------------------
[playwright] 
[playwright]     page.waitForSelector: Target closed
[playwright]     =========================== logs ===========================
[playwright]     waiting for selector ".quick-input-widget" to be visible
[playwright]     ============================================================
[playwright] 
[playwright]         at TheiaQuickCommandPalette.open (/Users/cjg/code/open/theia/examples/playwright/src/theia-quick-command-palette.ts:27:25)
[playwright] 
[playwright]   Slow test file: ../../tests/theia-quick-command.test.ts (4m)
[playwright]   Slow test file: ../../tests/theia-explorer-view.test.ts (2m)
[playwright]   Slow test file: ../../tests/theia-preference-view.test.ts (2m)
[playwright]   Slow test file: ../../tests/theia-problems-view.test.ts (2m)
[playwright]   Consider splitting slow test files to speed up parallel execution
[playwright] 
[playwright]   9 failed
[playwright]     ../../tests/theia-explorer-view.test.ts:46:9 › Theia Explorer View › should be possible to close and reopen it 
[playwright]     ../../tests/theia-explorer-view.test.ts:56:9 › Theia Explorer View › should show one folder named "sampleFolder" and one file named "sample.txt" 
[playwright]     ../../tests/theia-preference-view.test.ts:30:9 › Preference View › should be visible and active after being opened 
[playwright]     ../../tests/theia-preference-view.test.ts:37:9 › Preference View › should be able to read, set, and reset String preferences 
[playwright]     ../../tests/theia-problems-view.test.ts:30:9 › Theia Problems View › should be visible and active after being opened 
[playwright]     ../../tests/theia-problems-view.test.ts:37:9 › Theia Problems View › should be opened at the bottom and have the title 'Problems' 
[playwright]     ../../tests/theia-quick-command.test.ts:34:9 › Theia Quick Command › should show quick command palette 
[playwright]     ../../tests/theia-quick-command.test.ts:39:9 › Theia Quick Command › should trigger 'About' command after typing 
[playwright]     ../../tests/theia-quick-command.test.ts:53:9 › Theia Quick Command › should trigger 'Toggle Explorer View' command after typing 
[playwright]   10 skipped
[playwright]   29 passed (12m)

And a zip of the test-results folder:

test-results.zip

vince-fugnitto · 2022-03-15T17:41:28Z

Just to confirm that I also experienced issues with the tests on macOS using the following:

rm -rf ~/.theia - clean up any past configurations for the sake of testing
yarn
yarn browser build
yarn test:playwright

I get the following test summary:

[playwright]   10 failed
[playwright]     ../../tests/theia-explorer-view.test.ts:46:9 › Theia Explorer View › should be possible to close and reopen it
[playwright]     ../../tests/theia-explorer-view.test.ts:56:9 › Theia Explorer View › should show one folder named "sampleFolder" and one file named "sample.txt"
[playwright]     ../../tests/theia-main-menu.test.ts:50:9 › Theia Main Menu › should return menu item by name 'New File'
[playwright]     ../../tests/theia-preference-view.test.ts:30:9 › Preference View › should be visible and active after being opened
[playwright]     ../../tests/theia-preference-view.test.ts:37:9 › Preference View › should be able to read, set, and reset String preferences
[playwright]     ../../tests/theia-problems-view.test.ts:30:9 › Theia Problems View › should be visible and active after being opened
[playwright]     ../../tests/theia-problems-view.test.ts:37:9 › Theia Problems View › should be opened at the bottom and have the title 'Problems'
[playwright]     ../../tests/theia-quick-command.test.ts:34:9 › Theia Quick Command › should show quick command palette
[playwright]     ../../tests/theia-quick-command.test.ts:39:9 › Theia Quick Command › should trigger 'About' command after typing
[playwright]     ../../tests/theia-quick-command.test.ts:53:9 › Theia Quick Command › should trigger 'Toggle Explorer View' command after typing
[playwright]   10 skipped
[playwright]   28 passed (12m)

Additional Info

Node v12.14.1
macOS Catalina v10.15.7

ndoschek · 2022-03-16T08:57:46Z

Thanks for your feedback, I will have another look!

ndoschek · 2022-03-16T16:12:33Z

Thanks to @colin-grant-work 's screenshots I could track it down a little, and it seems that the alternative shortcut for opening the command palette (Control+Shift+p/Meta+Shift+p) did open the plugin view instead.
I adjusted that to use the other shortcut F1 to open the command palette.

I tested again with Ubuntu 20.04, Windows 11 and MacOS 12.2.1 and it worked on my side in multiple runs.
Could you please have another look if this solved your issues running the tests @colin-grant-work and @vince-fugnitto?
Thanks!

colin-grant-work · 2022-03-16T16:53:12Z

@ndoschek, I'm a bit suspicious of the fix, because Cmd+Shift+P always opens the command palette and never opens the plugin view when I run Theia in the browser on Mac (I've tried Firefox, Chrome, and Safari), but I'll run the tests with the new code and see how it goes.

colin-grant-work · 2022-03-16T17:00:28Z

With the new code, all the tests pass for me. 👍 Were so many tests failing because they depend on the command palette and the shortcut was failing, or were they failing because they were bailing after the first test of the command palette failed?

colin-grant-work · 2022-03-16T17:10:59Z

I'm still suspicious of the fix a bit: I brought back the OS check: await this.page.keyboard.press(OSUtil.isMacOS ? 'Meta+Shift+p' : 'Control+Shift+p');, and things still passed (which is good - passing is better than failing), so I'm not sure exactly what's changed to from passing to not passing.

ndoschek · 2022-03-17T10:01:08Z

@colin-grant-work Thanks a lot for re-testing so quickly!

Were so many tests failing because they depend on the command palette and the shortcut was failing, or were they failing because they were bailing after the first test of the command palette failed?

Yes, they depend on the command palette, as the generic function to open any TheiaView will do so via the command View: Open View....

so I'm not sure exactly what's changed to from passing to not passing

I also rebased the fix, but unfortunately I cannot tell either, why they failed for you in the first run (also, as I tested them on all three OSs, before and after).

If you agree, I would also prefer to stick to the initial shortcut (and hence to drop the additional commit).

vince-fugnitto · 2022-03-17T16:20:30Z

I confirmed that the tests now pass on macOS, I'm not sure what the difference is.

If you agree, I would also prefer to stick to the initial shortcut (and hence to drop the additional commit).

I confirmed that the original shortcut also works, I'm fine with reverting back to it.

colin-grant-work · 2022-03-17T16:23:35Z

I'm also in favor of reverting to the original shortcut 👍 . @vince-fugnitto, is it possible that it all had to do with global shortcuts somehow?

- Fix workspace path encoding for Windows - Stabilize workspace path fetching - Fix explorer review title button selectors - Introduce OSUtil helper functions - Fix shortcuts for MacOS - Improve opening of TheiaViews (speeds up tests by approx. 10s) Contributed on behalf of STMicroelectronics Signed-off-by: Nina Doschek <ndoschek@eclipsesource.com> Fixes eclipse-theia#10793

ndoschek · 2022-03-18T08:26:15Z

Thank you both for your feedback! 👍
I dropped the shortcut commit and rebased again.

ndoschek · 2022-03-18T08:50:32Z

@colin-grant-work, @vince-fugnitto
About the failing CI build - the tests are passing, but it failed due to some service shutting down.
Is there something I can do to solve this problem? Thanks in advance!

colin-grant-work · 2022-03-18T14:06:07Z

@ndoschek, it looks like CI is green (I suspect @vince-fugnitto re-ran the failing builds - unfortunately, the issues with Playwright on MacOS are not our only CI issues, but this PR will go some way to addressing at least one CI issue, so it's progress). Thanks for the contribution!

colin-grant-work

Looks good to me. The tests now pass on a variety of OS's, so they can be run by developers in various environments.

ndoschek mentioned this pull request Mar 2, 2022

playwright: tests fail on MacOS #10793

Closed

ndoschek force-pushed the issues/10793 branch from a1cae0e to 8e3b512 Compare March 2, 2022 08:40

vince-fugnitto added the playwright issues related to playwright tests label Mar 3, 2022

JonasHelming requested a review from colin-grant-work March 8, 2022 16:02

ndoschek force-pushed the issues/10793 branch from 8e3b512 to 82cc88d Compare March 16, 2022 16:08

ndoschek force-pushed the issues/10793 branch from 82cc88d to cb5f500 Compare March 18, 2022 08:24

colin-grant-work approved these changes Mar 18, 2022

View reviewed changes

colin-grant-work merged commit b3a5aab into eclipse-theia:master Mar 18, 2022

vince-fugnitto added this to the 1.24.0 milestone Mar 31, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

#10793 Fix playwright tests for Windows and MacOS #10826

#10793 Fix playwright tests for Windows and MacOS #10826

ndoschek commented Mar 2, 2022 •

edited

Loading

JonasHelming commented Mar 8, 2022

colin-grant-work commented Mar 11, 2022 •

edited

Loading

ndoschek commented Mar 11, 2022

colin-grant-work commented Mar 11, 2022

vince-fugnitto commented Mar 15, 2022

ndoschek commented Mar 16, 2022

ndoschek commented Mar 16, 2022

colin-grant-work commented Mar 16, 2022

colin-grant-work commented Mar 16, 2022

colin-grant-work commented Mar 16, 2022

ndoschek commented Mar 17, 2022

vince-fugnitto commented Mar 17, 2022

colin-grant-work commented Mar 17, 2022

ndoschek commented Mar 18, 2022

ndoschek commented Mar 18, 2022 •

edited

Loading

colin-grant-work commented Mar 18, 2022

colin-grant-work left a comment

#10793 Fix playwright tests for Windows and MacOS #10826

#10793 Fix playwright tests for Windows and MacOS #10826

Conversation

ndoschek commented Mar 2, 2022 • edited Loading

What it does

How to test

Review checklist

Reminder for reviewers

JonasHelming commented Mar 8, 2022

colin-grant-work commented Mar 11, 2022 • edited Loading

ndoschek commented Mar 11, 2022

colin-grant-work commented Mar 11, 2022

vince-fugnitto commented Mar 15, 2022

ndoschek commented Mar 16, 2022

ndoschek commented Mar 16, 2022

colin-grant-work commented Mar 16, 2022

colin-grant-work commented Mar 16, 2022

colin-grant-work commented Mar 16, 2022

ndoschek commented Mar 17, 2022

vince-fugnitto commented Mar 17, 2022

colin-grant-work commented Mar 17, 2022

ndoschek commented Mar 18, 2022

ndoschek commented Mar 18, 2022 • edited Loading

colin-grant-work commented Mar 18, 2022

colin-grant-work left a comment

Choose a reason for hiding this comment

ndoschek commented Mar 2, 2022 •

edited

Loading

colin-grant-work commented Mar 11, 2022 •

edited

Loading

ndoschek commented Mar 18, 2022 •

edited

Loading