[EuiInMemoryTable] Syntax error when searching with special characters

[cee-chen]

Problem

See elastic/kibana#138499

The security team is using our in memory table's built-in search API for what's essentially plain text search. Unfortunately, some of the strings they're searching for have special characters in them, and the underlying EuiSearchBar component that the table uses throws a syntax error when the search string contains a special character.

Possible solutions

1. Addressing within EuiSearchBar: one approach could be something similar to what was done in [EuiSearchBar] Automatically wrap special characters (that aren't used in EQL syntax) in quotes #6356 (NOTE: this auto-escaping was only performed for terms/types and not for top-level searches).

2. Addressing within EuiInMemoryTable: essentially the use-case for the above Kibana issue doesn't need the EQL behavior of EuiSearchBar whatsoever. We could consider adding a prop to the search obj that allows consumers to specify they simply want to use a plaintext EuiFieldSearch component instead of the full-blown EQL syntax, and search based on that.

Timeline

Hoping to pick this up during my support week this week, but if not, we should triage priority at our next grooming meeting

[cee-chen]

@janmonschke I'm experimenting with auto-escaping more special characters in EuiSearchBar under the hood. Unfortunately, the following characters will never be correctly escaped/handled by the search bar, because they're special characters used to interpret EQL: ", (), : and \.

Just curious, what do you think is the likelihood of your team's products having items with the above characters? If the answer is likely, we'll probably need to look into avoiding EuiSearchBar/EQL completely and figuring out some other way of iterating/searching over all provided items.

[janmonschke]

(),

Those might be the only ones that I can think of. But at that point it's become such an edge case, that I think we'll see the 'false positive' error very rarely :)

[cee-chen]

Good news - I'm down to just \ throwing errors/breaking EQL (EDIT: Thanks to Tomasz, every special character now works! 🎉 ) :) #7175 if you're curious to check it out ahead of time!

Note: It's still using EQL under the hood to execute over every item, which perhaps isn't as performant as it could be, but it's at least not less performant than current behavior.

[cee-chen]

Just as a quick update - the upcoming API to enable searching for special characters is:

<EuiInMemoryTable
  search={true}
  searchFormat="text"
  // ... other props
/>

Please note that filters will not work at all when using this search format, so this truly should only be used for plain text search.

[cee-chen]

🤦 Ahhh, shoot. I just noticed the screenshots in the linked Kibana issue have an "only favorites" filter, which will stop working if used in conjunction with searchFormat="text". So, @janmonschke I don't think your team can use this new feature after all...

I'm experimenting with auto-escaping more special characters in EuiSearchBar under the hood. Unfortunately, the following characters will never be correctly escaped/handled by the search bar, because they're special characters used to interpret EQL: ", (), : and \.

If your team wants to preserve the "Only favorites" filter, this has to be the approach taken. We simply cannot parse the above special characters if EQL for filtering is also being used.