Add method to wait for loading to complete in Add Panel table

[LeeDr]

Fixes #20819

The new euiBasicTable is used in the Dashboard Add Panel flyout. When it opens it queries for the list of visualizations to show in the table. It shows a moving blue progress bar during that time. If the test tries to filter the visualizations by entering text into the savedObjectFinderSearchInput before it's ready it can fail (see issue above).

This PR changes 3 things;

1. adds a new method waitForEuiTableLoading which waits does waitForDeletedByClassName('euiBasicTable-loading')

2. calls that wait method in filterEmbeddableNames(name)

3. puts the testSubjects.setValue('savedObjectFinderSearchInput', name); inside a try loop. This change by itself should probably be enough to fix the issue, but I'd rather wait for the loading indicator than see tryForTime failures in the log.

[elasticmachine]

💔 Build Failed

• continuous-integration/kibana-ci/pull-request

[LeeDr]

There was a failure in an ML x-pack test. I pinged the ML team but it's late in the day for them now. I'm going to have jenkins test this again just in case it's intermittent or something. Otherwise we might have to skip or ignore the failing test.

[elasticmachine]

💔 Build Failed

• continuous-integration/kibana-ci/pull-request

[stacey-gammon]

I don't think this extra retry loop adds anything because there is already a retry loop on the inside of setValue:

async setValue(selector, text) {
      return await retry.try(async () => {
        await this.click(selector);
        // in case the input element is actually a child of the testSubject, we
        // call clearValue() and type() on the element that is focused after
        // clicking on the testSubject
        const input = await remote.getActiveElement();
        await input.clearValue();
        await input.type(text);
      });
    }

We could think about confirming that the value was actually set by adding something in that function call that does like:

async setValue(selector, text) {
      return await retry.try(async () => {
        await this.click(selector);
        // in case the input element is actually a child of the testSubject, we
        // call clearValue() and type() on the element that is focused after
        // clicking on the testSubject
        const input = await remote.getActiveElement();
        await input.clearValue();
        await input.type(text);
        const newValue = await this.getVisibleText(selector);
         if (newValue != text) {
           throw new Error('Value was not set');
          } 
      });
    }

But I'm not sure if getVisibleText is what we'd want or if it's work on all element types (might be something like getValue we should use?).

[LeeDr]

I didn't realize that setValue method had a retry loop. But that makes me wonder why it wasn't solving this problem from the beginning. The first error in this case was coming from the clearValue() step in that setValue method;
invalid element state: Element must be user-editable in order to clear it.

But then the next message says that it failed to find the element.
no such element: Unable to locate element: {"method":"css selector","selector":"[data-test-subj~="savedObjectFinderSearchInput"]"}

And when the test finally fails and takes a screenshot, we see that it has navigated away from the Add Panel flyout and has opened the New visualization page (which might explain why it can no longer find the element). It seems as if this inner-most retry isn't staying in this loop for some reason, and an outer retry is trying to clicking on "Add" again, or somewhere else that causes the incorrect navigation.

What you don't see here is the debug logging I had in place which did show that the euiBasicTable did initially show the euiBasicTable-loading class every time we opened the Add Panel flyout and that it was gone after we waited for it to be deleted;

         │ debg  DashboardAddPanel.isAddPanelOpen
         │ debg  TestSubjects.exists(dashboardAddPanel)
         │ debg  existsByDisplayedByCssSelector [data-test-subj~="dashboardAddPanel"]
         │ debg  TestSubjects.find(dashboardAddPanel)
         │ debg  findByCssSelector [data-test-subj~="dashboardAddPanel"]
         │ debg  ################################################################# euiBasicTable euiBasicTable-loading
         │ debg  ##############################++++++++++++++++++++++++++++++++++++++ euiBasicTable
         │ debg  TestSubjects.find(savedObjectFinderSearchInput)
         │ debg  findByCssSelector [data-test-subj~="savedObjectFinderSearchInput"]
         │ debg  isGlobalLoadingIndicatorVisible
         │ debg  TestSubjects.exists(globalLoadingIndicator)

[stacey-gammon]

Yea, so my suspicion is that the mechanism for typing selenium uses is accidentally hitting the Add Visualization button. Once the page navigates away, doesn't matter how many times it loops, it's going to fail. That's why I think we have to add a function that waits till the flyout finishes it's transition before we even start typing into that field.

[stacey-gammon]

Seems like this function could go somewhere more general as it might be useful for other places that use the eui table and want to wait for it to load. I don't see a good spot for it right now. Maybe a new service named euiComponents? Could take in an optional selector name to use to narrow down but I wonder if you can use it without an element, directly on remote (just like we do for the global loading indicator - making sure it doesn't exist at all, anywhere on the page).

[LeeDr]

I thought about that. But the euiBasicTable can get populated from any data source and doesn't necessarily require a request/response to Elasticsearch so I'm not sure what to do.

Again, the retry loop in setValue seems like it should always handle the case where the field isn't initially user-editable.

[nreese]

Seems like this function could go somewhere more general as it might be useful for other places that use the eui table and want to wait for it to load. I don't see a good spot for it right now. Maybe a new service named euiComponents?

There is a flyout service and I have a PR out for a combo box service. #21219.

[stacey-gammon]

I like the addition of the waitForEuiTableLoading, but I don't know if that will solve this particular issue (or all of them anyway - might solve one).

I think we need to find a way to ensure that the flyout has finished it's transition from left to right. Could maybe add a function to testSubjects and the find service called waitForTransitionEnd that in a loop checks an elements x/y position to see if it's changing on the scree (maybe with a teeny wait in the middle, like 250 ms). Once the two match, return. Unless there is a better CSS way to check if a transition is in progress or not, but I'm not aware.

[LeeDr]

retest

[LeeDr]

@stacey-gammon I'm not sure the flying-out action has any impact on this problem. It's not like we ever see a failure screenshot with the fly-out half way out?
But I was seeing the error that the savedObjectFinderSearchInput was not user-editable, and that does seem directly related to the euiBasicTable-loading class on the table.

In my test runs, I haven't seen any retry loops get hit in this area of code. Not the inner-most one in setValue, not the new one I added in filterEmbeddableNames, and not any outer ones (which I think cause the incorrect navigation). So that makes me think that waiting for the table loading to complete solves the flakiness of some of these tests.

[stacey-gammon]

It's not like we ever see a failure screenshot with the fly-out half way out?

Well that's cause the visualization page would finish loading during the retry. I have seen the visualization page screenshot which never made sense if it was just adding panels to a dashboard.

that makes me think that waiting for the table loading to complete solves the flakiness of some of these tests.

I'm happy to check this in to solve the flakiness of some of these tests at least, and the table load wait makes sense to me. I just still suspect another issue with the flyout transition. But we can check this in first and see what happens with the other flaky issues.

[LeeDr]

@stacey-gammon I had a thought. If I temporarily tweaked the fly-out time so that it went really slow, like 2 or 3 seconds to fly-out so that the table loading had already finished before the fly-out was even half way out. Then we could see if that matters or not. I'm guessing it won't matter, but I've been wrong before.
Is there an easy place I can tweak that speed?

[stacey-gammon]

@LeeDr, what is your goal by doing that? are you trying to hit the bug or see if it solves it? If the latter, you could try to completely remove the transition and see if that fixes it. If you mean the former... you could try it but I don't think it'd offer an obvious answer since it's flaky. Well, neither would give obvious answers since it's flaky.

I think perhaps step one would be to see if a single test failure can be consistently reproduced by modifying the code so it jenkins runs a single test suite a bunch of times. Historically I've done this like this: c09460f

Though given how many other flaky tests there are, may also need to pair that with commenting out some of the other tests to focus only on fixing a single issue at a time.

[elasticmachine]

💔 Build Failed

• continuous-integration/kibana-ci/pull-request

[elasticmachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request

[elasticmachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request

[stacey-gammon]

btw @LeeDr - haven't had time to investigate more but I noticed while running the tests locally that there is a significant wait (couple seconds) right before typing into the add panel. I'm thinking it might be from this PR. I wonder if that is why it appeared to fix the open flyout issue. Maybe it didn't fix the root cause but just fixed it because it was equivalent to adding a long sleep.

Anyway, as long as tests are not flaky, that's an improvement, but might be worth circling back around to this at some point in the future.