[ES|QL] open suggestions automatically in sources lists and `ENRICH` #191312

drewdaemon · 2024-08-26T19:44:17Z

Summary

Also, a follow-on to #187184 with certain source names (e.g. foo$bar, my-policy) and fields in the ENRICH command.

During this effort I discovered #191321 and a bug with the validation of field names in the "ENRICH ... WITH" list (documented here), both of which will be addressed separately.

Sources

General flow

Screen.Recording.2024-08-28.at.2.22.32.PM.mov

Works with wild-card matches

Screen.Recording.2024-08-26.at.2.38.55.PM.mov

`METADATA` field list

Screen.Recording.2024-08-26.at.4.06.40.PM.mov

ENRICH

Autosuggest now helps you along

Screen.Recording.2024-08-28.at.12.57.32.PM.mov

Also, fixed this bug (follow on to #187184 )

Screen.Recording.2024-08-28.at.12.36.51.PM.mov

Checklist

Unit or functional tests were updated or added to match the most common scenarios

drewdaemon · 2024-08-28T23:27:40Z

/ci

drewdaemon · 2024-08-29T01:25:48Z

/ci

elasticmachine · 2024-08-29T03:16:08Z

Pinging @elastic/kibana-esql (Team:ESQL)

drewdaemon · 2024-08-29T03:16:49Z

@elasticmachine merge upstream

qn895 · 2024-09-10T14:39:38Z

packages/kbn-esql-validation-autocomplete/src/autocomplete/autocomplete.test.ts

@@ -845,6 +961,57 @@ describe('autocomplete', () => {
      '| ',
    ]);

+    describe('ENRICH', () => {


I think it's worth splitting these new blocks out to new files 🤔

How would you split it up?

Initially, I suggested this because there are two separate describe('enrich') blocks in this file. But one is to test the actual suggestions for enrich, and one is to test advancing the cursor and opening the suggestion menu automatically. We can discuss this in office hours how best to structure these autocomplete tests, but imo it might be worth moving the big blocks like advancing the cursor into their own files in the future.

Exactly. There's a tension between splitting things up by command or by language feature across commands.

I'm not sure what is best but, yes, worth a discussion!

qn895 · 2024-09-10T14:43:12Z

packages/kbn-esql-validation-autocomplete/src/shared/helpers.ts

+ * @returns
+ */
+export function findFinalWord(text: string) {
+  const words = text.split(/\s+/);


For the regex in this and in findPreviousWord, is it worth trimming the text first? For example "foo bar " would give ['', 'foo', 'bar', ''].

Oh ok I see now we do it expect it to be an empty space in some cases.

qn895 · 2024-09-10T14:46:55Z

packages/kbn-esql-validation-autocomplete/src/autocomplete/factories.ts

@@ -272,7 +273,7 @@ export const buildPoliciesDefinitions = (
 ): SuggestionRawDefinition[] =>
  policies.map(({ name: label, sourceIndices }) => ({
    label,
-    text: getSafeInsertText(label, { dashSupported: true }),
+    text: getSafeInsertText(label, { dashSupported: true }) + ' ',


I'm starting to think we should make this ' ' a constant for clarity and easy searching. So something like OPEN_SUGGESTION = ' ' and then text + OPEN_SUGGESTION

The ' ' is not what opens the suggestion menu. That is the command property.

Sorry, I meant advance cursor. Switched it around in my brain. And this is just a thought, not a blocker.

qn895 · 2024-09-10T15:27:36Z

packages/kbn-esql-validation-autocomplete/src/autocomplete/__tests__/helpers.ts

-      // ok, this is a bit of cheating as it's using the same logic as in the helper
-      enrichFields.map((field) => (/[^a-zA-Z\d_\.@]/.test(field) ? `\`${field}\`` : field))
-    );
+    .flatMap(({ enrichFields }) => enrichFields.map((field) => getSafeInsertText(field)));


nit: we can probably use enrichFields.map(getSafeInsertText) here

…s-for-sources

drewdaemon · 2024-09-10T17:34:56Z

@elasticmachine merge upstream

qn895 · 2024-09-10T20:04:48Z

LGTM 🎉

drewdaemon · 2024-09-10T20:10:27Z

@elasticmachine merge upstream

drewdaemon · 2024-09-13T02:23:16Z

@elasticmachine merge upstream

stratoula

As we have blocked this PR till we have this #187247 I am blocking also with Request changes (just to ensure that it wont be accidentally being merged). It is coming #192631 🎉

We can also mark it as draft for now.

drewdaemon · 2024-09-14T22:09:06Z

@elasticmachine merge upstream

drewdaemon · 2024-09-14T22:09:44Z

I'm going to keep it out of draft mode since it will be unblocked when #192953 merges

stratoula

Unblocking the PR now that the blocker got resolved. LGTM, I did a brief testing locally to ensure that it works as expected.

kibana-ci · 2024-09-16T11:06:25Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 8569689

Failed CI Steps

Jest Tests #1

Test Failures

[job] [logs] Jest Tests #1 / SolutionFilter when the owner is a single solution renders options correctly

Metrics [docs]

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
`@kbn/esql-validation-autocomplete`	10	11	+1

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`kbnUiSharedDeps-srcJs`	3.4MB	3.4MB	+656.0B

History

💛 Build #234225 was flaky fc67532
💚 Build #233896 succeeded 69085d8
💛 Build #233243 was flaky df61634
💔 Build #233204 failed ff3fc5a
💔 Build #233180 failed fbe444e
💛 Build #232802 was flaky fc48c2a

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

…lastic#191312) ## Summary Part of elastic#189662 Also, a follow-on to elastic#187184 with certain source names (e.g. `foo$bar`, `my-policy`) and fields in the `ENRICH` command. During this effort I discovered elastic#191321 and a bug with the validation of field names in the "ENRICH ... WITH" list (documented [here](elastic#177699)), both of which will be addressed separately. ## Sources ### General flow https://github.com/user-attachments/assets/4b103621-0e66-4c36-807f-4932f0cb8faf ### Works with wild-card matches https://github.com/user-attachments/assets/6b47fffc-e922-4e2d-b6aa-3d9a2fc2236c ### `METADATA` field list https://github.com/user-attachments/assets/d3bdf4dc-1d0c-4d56-81d7-af6bc4e25a4a ## ENRICH Autosuggest now helps you along https://github.com/user-attachments/assets/d627484c-e729-4dc7-9e7b-795395a31d4f Also, fixed this bug (follow on to elastic#187184 ) https://github.com/user-attachments/assets/aa62a0c3-6db5-434a-829a-59f14c5c4c85 ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co> (cherry picked from commit e404a39)

kibanamachine · 2024-09-16T13:46:20Z

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

…60;ENRICH` (#191312) (#193013) # Backport This will backport the following commits from `main` to `8.x`: - [[ES|QL] open suggestions automatically in sources lists and `ENRICH` (#191312)](#191312)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)  Co-authored-by: Drew Tate <drew.tate@elastic.co>

…lastic#191312) ## Summary Part of elastic#189662 Also, a follow-on to elastic#187184 with certain source names (e.g. `foo$bar`, `my-policy`) and fields in the `ENRICH` command. During this effort I discovered elastic#191321 and a bug with the validation of field names in the "ENRICH ... WITH" list (documented [here](elastic#177699)), both of which will be addressed separately. ## Sources ### General flow https://github.com/user-attachments/assets/4b103621-0e66-4c36-807f-4932f0cb8faf ### Works with wild-card matches https://github.com/user-attachments/assets/6b47fffc-e922-4e2d-b6aa-3d9a2fc2236c ### `METADATA` field list https://github.com/user-attachments/assets/d3bdf4dc-1d0c-4d56-81d7-af6bc4e25a4a ## ENRICH Autosuggest now helps you along https://github.com/user-attachments/assets/d627484c-e729-4dc7-9e7b-795395a31d4f Also, fixed this bug (follow on to elastic#187184 ) https://github.com/user-attachments/assets/aa62a0c3-6db5-434a-829a-59f14c5c4c85 ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>

drewdaemon added 2 commits August 26, 2024 12:40

refactor integration suggestions in preparation

22e9bc6

basic sources support

246600e

drewdaemon added Feature:ES|QL ES|QL related features in Kibana Team:ESQL ES|QL related features in Kibana labels Aug 26, 2024

drewdaemon added 7 commits August 26, 2024 14:42

Support wild-cards in index name

0bb1170

update some tests

dc69235

Support METADATA field list

050b040

fix up ENRICH experience

5376a57

include replacement range for policy names

c298abb

add replacement range for ENRICH

b52fdc4

fix suggestions for subsequent sources

b18784a

drewdaemon mentioned this pull request Aug 28, 2024

[ES|QL] sources are suggested over and over #191321

Open

drewdaemon added release_note:skip Skip the PR/issue when compiling release notes release_note:enhancement and removed release_note:skip Skip the PR/issue when compiling release notes labels Aug 28, 2024

drewdaemon mentioned this pull request Aug 28, 2024

[ES|QL] menu should open again automatically when users accept suggestions #189662

Closed

drewdaemon added 5 commits August 28, 2024 16:11

update values test

102116b

test suite

797e4c9

test

f31bc05

fix values suggestions

e298e70

restore metadata field, <suggest> behavior

0caf743

update the lil util

cbaf4f2

drewdaemon changed the title ~~[ES|QL] open suggestions automatically in sources list~~ [ES|QL] open suggestions automatically in sources lists and ENRICH Aug 29, 2024

drewdaemon marked this pull request as ready for review August 29, 2024 03:16

drewdaemon requested a review from a team as a code owner August 29, 2024 03:16

Merge branch 'main' into open-suggestions-for-sources

a11d1fc

qn895 reviewed Sep 10, 2024

View reviewed changes

Merge branch 'main' of github.com:elastic/kibana into open-suggestion…

fbe444e

…s-for-sources

Merge branch 'main' into open-suggestions-for-sources

ff3fc5a

qn895 approved these changes Sep 10, 2024

View reviewed changes

Merge branch 'main' into open-suggestions-for-sources

df61634

Merge branch 'main' into open-suggestions-for-sources

69085d8

stratoula requested changes Sep 13, 2024

View reviewed changes

Merge branch 'main' into open-suggestions-for-sources

fc67532

Merge branch 'main' into open-suggestions-for-sources

8569689

stratoula added backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) and removed blocked labels Sep 16, 2024

stratoula approved these changes Sep 16, 2024

View reviewed changes

drewdaemon merged commit e404a39 into elastic:main Sep 16, 2024
23 checks passed

kibanamachine added the v9.0.0 label Sep 16, 2024

kibanamachine mentioned this pull request Sep 16, 2024

[8.x] [ES|QL] open suggestions automatically in sources lists and `ENRICH` (#191312) #193013

Merged

kibanamachine added the v8.16.0 label Sep 16, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ES|QL] open suggestions automatically in sources lists and `ENRICH` #191312

[ES|QL] open suggestions automatically in sources lists and `ENRICH` #191312

drewdaemon commented Aug 26, 2024 •

edited by kibanamachine

Loading

drewdaemon commented Aug 28, 2024

drewdaemon commented Aug 29, 2024

elasticmachine commented Aug 29, 2024

drewdaemon commented Aug 29, 2024

qn895 Sep 10, 2024

drewdaemon Sep 10, 2024

qn895 Sep 10, 2024

drewdaemon Sep 10, 2024

drewdaemon Sep 10, 2024

qn895 Sep 10, 2024

qn895 Sep 10, 2024 •

edited

Loading

qn895 Sep 10, 2024

drewdaemon Sep 10, 2024

qn895 Sep 10, 2024

qn895 Sep 10, 2024 •

edited

Loading

drewdaemon commented Sep 10, 2024

qn895 commented Sep 10, 2024

drewdaemon commented Sep 10, 2024

drewdaemon commented Sep 13, 2024

stratoula left a comment •

edited

Loading

drewdaemon commented Sep 14, 2024

drewdaemon commented Sep 14, 2024

stratoula left a comment

kibana-ci commented Sep 16, 2024

kibanamachine commented Sep 16, 2024

[ES|QL] open suggestions automatically in sources lists and ENRICH #191312

[ES|QL] open suggestions automatically in sources lists and ENRICH #191312

Conversation

drewdaemon commented Aug 26, 2024 • edited by kibanamachine Loading

Summary

Sources

General flow

Works with wild-card matches

METADATA field list

ENRICH

Checklist

drewdaemon commented Aug 28, 2024

drewdaemon commented Aug 29, 2024

elasticmachine commented Aug 29, 2024

drewdaemon commented Aug 29, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qn895 Sep 10, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qn895 Sep 10, 2024 • edited Loading

Choose a reason for hiding this comment

drewdaemon commented Sep 10, 2024

qn895 commented Sep 10, 2024

drewdaemon commented Sep 10, 2024

drewdaemon commented Sep 13, 2024

stratoula left a comment • edited Loading

Choose a reason for hiding this comment

drewdaemon commented Sep 14, 2024

drewdaemon commented Sep 14, 2024

stratoula left a comment

Choose a reason for hiding this comment

kibana-ci commented Sep 16, 2024

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

Public APIs missing exports

Page load bundle

History

kibanamachine commented Sep 16, 2024

💚 All backports created successfully

Questions ?

[ES|QL] open suggestions automatically in sources lists and `ENRICH` #191312

[ES|QL] open suggestions automatically in sources lists and `ENRICH` #191312

drewdaemon commented Aug 26, 2024 •

edited by kibanamachine

Loading

`METADATA` field list

qn895 Sep 10, 2024 •

edited

Loading

qn895 Sep 10, 2024 •

edited

Loading

stratoula left a comment •

edited

Loading