Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enabling Uptime and Dashboard a11y test #91017

Merged
merged 9 commits into from
Feb 20, 2021
Merged

Enabling Uptime and Dashboard a11y test #91017

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
953ada5
enable uptime a11y test
Feb 10, 2021
8639776
Merge remote-tracking branch 'upstream/master' into 90555
Feb 17, 2021
f3a711b
fixing a11y tests
Feb 17, 2021
c153e93
remove testing function
Feb 17, 2021
df7dc21
fixing flakiness
Feb 18, 2021
c9084c0
fix dashboard tests
Feb 18, 2021
30b6cf7
swallow error messages
Feb 19, 2021
bc62d98
Merge remote-tracking branch 'upstream/master' into 90555
Feb 19, 2021
572b2fc
revert common_page object toasts handling
Feb 19, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion test/accessibility/services/a11y/analyze_with_axe.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,19 @@ export function analyzeWithAxe(context, options, callback) {
selector: '[data-test-subj="comboBoxSearchInput"] *',
},
{
// EUI bug: https://github.com/elastic/eui/issues/4474
id: 'aria-required-parent',
selector: '[class=*"euiDataGridRowCell"][role="gridcell"] ',
selector: '[class=*"euiDataGridRowCell"][role="gridcell"]',
},
{
// 3rd-party library; button has aria-describedby
id: 'button-name',
selector: '[data-rbd-drag-handle-draggable-id]',
},
{
// EUI bug: https://github.com/elastic/eui/issues/4536
id: 'duplicate-id',
selector: '.euiSuperDatePicker *',
Comment on lines +34 to +46
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ignore all 3rd party errors because there's nothing we can do directly in Kibana to resolve them until they're resolved upstream.

},
],
});
Expand Down
2 changes: 1 addition & 1 deletion test/functional/services/dashboard/panel_actions.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ export function DashboardPanelActionsProvider({ getService, getPageObjects }: Ft

async clickExpandPanelToggle() {
log.debug(`clickExpandPanelToggle`);
this.openContextMenu();
await this.openContextMenu();
const isActionVisible = await testSubjects.exists(TOGGLE_EXPAND_PANEL_DATA_TEST_SUBJ);
if (!isActionVisible) await this.clickContextMenuMoreItem();
await testSubjects.click(TOGGLE_EXPAND_PANEL_DATA_TEST_SUBJ);
Expand Down
15 changes: 13 additions & 2 deletions test/functional/services/toasts.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,10 +45,21 @@ export function ToastsProvider({ getService }: FtrProviderContext) {
public async dismissAllToasts() {
const list = await this.getGlobalToastList();
const toasts = await list.findAllByCssSelector(`.euiToast`);

if (toasts.length === 0) return;

for (const toast of toasts) {
await toast.moveMouseTo();
const dismissButton = await testSubjects.findDescendant('toastCloseButton', toast);
await dismissButton.click();

if (await testSubjects.descendantExists('toastCloseButton', toast)) {
try {
const dismissButton = await testSubjects.findDescendant('toastCloseButton', toast);
await dismissButton.click();
} catch (err) {
// ignore errors
// toasts are finnicky because they can dismiss themselves right before you close them
}
}
}
}

Expand Down
87 changes: 28 additions & 59 deletions x-pack/test/accessibility/apps/dashboard_edit_panel.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,9 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) {
const PageObjects = getPageObjects(['security', 'common']);
const toasts = getService('toasts');

// Failing: See https://github.com/elastic/kibana/issues/91592
describe.skip('Dashboard Edit Panel', () => {
const PANEL_TITLE = 'Visualization PieChart';

describe('Dashboard Edit Panel', () => {
before(async () => {
await esArchiver.load('dashboard/drilldowns');
await esArchiver.loadIfNeeded('logstash_functional');
Expand All @@ -33,100 +34,68 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) {
await esArchiver.unload('dashboard/drilldowns');
});

// embeddable edit panel
it(' A11y test on dashboard edit panel menu options', async () => {
await testSubjects.click('embeddablePanelToggleMenuIcon');
it('can open menu', async () => {
await dashboardPanelActions.openContextMenu();
await a11y.testAppSnapshot();
});

// https://github.com/elastic/kibana/issues/77931
it.skip('A11y test for edit visualization and save', async () => {
await testSubjects.click('embeddablePanelToggleMenuIcon');
await testSubjects.click('embeddablePanelAction-editPanel');
await testSubjects.click('visualizesaveAndReturnButton');
it('can clone panel', async () => {
await dashboardPanelActions.clonePanelByTitle(PANEL_TITLE);
await a11y.testAppSnapshot();
await toasts.dismissAllToasts();
await dashboardPanelActions.removePanelByTitle(`${PANEL_TITLE} (copy)`);
});

// clone panel
it(' A11y test on dashboard embeddable clone panel', async () => {
await testSubjects.click('embeddablePanelAction-clonePanel');
it('can customize panel', async () => {
await dashboardPanelActions.customizePanel();
await a11y.testAppSnapshot();
await toasts.dismissAllToasts();
await dashboardPanelActions.removePanelByTitle('Visualization PieChart (copy)');
});

// edit panel title
it(' A11y test on dashboard embeddable edit dashboard title', async () => {
await testSubjects.click('embeddablePanelToggleMenuIcon');
await testSubjects.click('embeddablePanelAction-ACTION_CUSTOMIZE_PANEL');
await a11y.testAppSnapshot();
await testSubjects.click('customizePanelHideTitle');
it('can hide panel title', async () => {
await dashboardPanelActions.clickHidePanelTitleToggle();
await a11y.testAppSnapshot();
await testSubjects.click('saveNewTitleButton');
});

// create drilldown
it('A11y test on dashboard embeddable open flyout and drilldown', async () => {
it('can drilldown', async () => {
await testSubjects.click('embeddablePanelToggleMenuIcon');
await testSubjects.click('embeddablePanelAction-OPEN_FLYOUT_ADD_DRILLDOWN');
await a11y.testAppSnapshot();
await testSubjects.click('flyoutCloseButton');
});

// clicking on more button
it('A11y test on dashboard embeddable more button', async () => {
await testSubjects.click('embeddablePanelToggleMenuIcon');
await testSubjects.click('embeddablePanelMore-mainMenu');
it('can view more actions', async () => {
await dashboardPanelActions.openContextMenuMorePanel();
await a11y.testAppSnapshot();
});

// https://github.com/elastic/kibana/issues/77422
it.skip('A11y test on dashboard embeddable custom time range', async () => {
await testSubjects.click('embeddablePanelToggleMenuIcon');
it('can create a custom time range', async () => {
await dashboardPanelActions.openContextMenuMorePanel();
await testSubjects.click('embeddablePanelAction-CUSTOM_TIME_RANGE');
await a11y.testAppSnapshot();
await testSubjects.click('addPerPanelTimeRangeButton');
});

// flow will change whenever the custom time range a11y issue gets fixed.
// Will need to click on gear icon and then click on more.

// inspector panel
it('A11y test on dashboard embeddable open inspector', async () => {
await testSubjects.click('embeddablePanelAction-openInspector');
it('can open inspector', async () => {
await dashboardPanelActions.openInspector();
await a11y.testAppSnapshot();
await testSubjects.click('euiFlyoutCloseButton');
});

// fullscreen
it('A11y test on dashboard embeddable fullscreen', async () => {
await testSubjects.click('embeddablePanelToggleMenuIcon');
await testSubjects.click('embeddablePanelMore-mainMenu');
await testSubjects.click('embeddablePanelAction-togglePanel');
await a11y.testAppSnapshot();
});

// minimize fullscreen panel
it('A11y test on dashboard embeddable fullscreen minimize ', async () => {
await testSubjects.click('embeddablePanelToggleMenuIcon');
await testSubjects.click('embeddablePanelMore-mainMenu');
await testSubjects.click('embeddablePanelAction-togglePanel');
it('can go fullscreen', async () => {
await dashboardPanelActions.clickExpandPanelToggle();
await a11y.testAppSnapshot();
await dashboardPanelActions.clickExpandPanelToggle();
});

// replace panel
it('A11y test on dashboard embeddable replace panel', async () => {
await testSubjects.click('embeddablePanelToggleMenuIcon');
await testSubjects.click('embeddablePanelMore-mainMenu');
await testSubjects.click('embeddablePanelAction-replacePanel');
it('can replace panel', async () => {
await dashboardPanelActions.replacePanelByTitle();
await a11y.testAppSnapshot();
await testSubjects.click('euiFlyoutCloseButton');
});

// delete from dashboard
it('A11y test on dashboard embeddable delete panel', async () => {
await testSubjects.click('embeddablePanelToggleMenuIcon');
await testSubjects.click('embeddablePanelMore-mainMenu');
await testSubjects.click('embeddablePanelAction-deletePanel');
it('can delete panel', async () => {
await dashboardPanelActions.removePanel();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use team-maintained page objects if available because sometimes "removing a panel" is more complicated than it seems

await a11y.testAppSnapshot();
});
});
Expand Down
9 changes: 6 additions & 3 deletions x-pack/test/accessibility/apps/uptime.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,9 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) {
const uptimeService = getService('uptime');
const esArchiver = getService('esArchiver');
const es = getService('es');
const toasts = getService('toasts');

// FLAKY: https://github.com/elastic/kibana/issues/90555
// Failing: See https://github.com/elastic/kibana/issues/90555
describe.skip('uptime', () => {
describe('uptime', () => {
before(async () => {
await esArchiver.load('uptime/blank');
await makeChecks(es, A11Y_TEST_MONITOR_ID, 150, 1, 1000, {
Expand Down Expand Up @@ -61,7 +60,11 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) {

it('overview alert popover controls', async () => {
await uptimeService.overview.openAlertsPopover();
await toasts.dismissAllToasts();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whenever you see an error like #90555 it's probably a toasts issue, even if you're actively opening/closing popovers/flyouts.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯

await a11y.testAppSnapshot();
});

it('overview alert popover controls nested content', async () => {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Each snapshot should go into its own test just to make identifying the point of failure easier.

The test name is always easy to find but figuring out where within a test you are can be difficult and axe is basically a black box.

await uptimeService.overview.navigateToNestedPopover();
await a11y.testAppSnapshot();
});
Expand Down