[Meta] Discover rewrite to EUI

[timroes]

While in the process of moving over Discover to the new platform (#38242) the largest part is rewriting Discover in EUI/React. During that rewrite we're trying to address many open issues/enhancement if possible to make sure we're starting with a good new experience.

For the actual table component the new datagrid component in EUI (elastic/eui#1689) will be used, which would hopefully already solve a lot of issues.

This meta issue tracks important issues we are trying to address while rewriting:

More interactions

• Resizable columns (Resizable discover table columns #737) via EUI datagrid
• Allow sorting on multiple fields (Allow sorting on multiple fields #696) UI via EUI datagrid (still required additional logic)
• Frozen columns/rows ([Discover] Lock column when scrolling discover table to the right #12439/Make doc table header static #12417) via EUI datagrid
• Drag'n'drop reordering of columns (Drag-and-drop to re-order Discover table columns #8706) via EUI datagrid
• Easier to expand row (Double-click to expand a row #37096)

More configuration/styling

• Hide histogram (Hide histogram on Discover page #6164)
• Individual column configuration
  • Individual column (field) formatters (e.g. Allow multiple field formatters on one field/per visualization #16821, Make Doc link in Discovery app plug-able #10761)
  • Truncation length
• (Conditional) row highlighting (Highlight row in Discover instead of just the value #36592)

Remaining Tasks

• Use EUIDataGrid for the data table: [Discover] Migrate Discover Table and Discover Saved Search to use EUIDataGrid component #61906
• Saved search on dashboard. Should be in the same minor as a Datagrid & tracked n the issue above.
• De-angularize and EUI-ficate sidebar: [Discover] Deangularize and euificate sidebar  #47559
• Migrate Chart - still mostly angular
• Migrate Discover context view - uses legacy doc table, all angular
• Discover doc viewer

[elasticmachine]

Pinging @elastic/kibana-app

[AlonaNadler]

Few things which we should add to the list :

• Ability to set a custom field name in Discovery and Saved Searches Ability to set a custom field name in Discovery and Saved Searches #15246
• Evaluate if the side menu can be similar to Lens, provide consistency and the ability to see field context, except for the dragging of fields it can share the same capabilities
  
• In addition to highlighting the entire row Highlight row in Discover instead of just the value #36592, Discover can benefit from highlighting capabilities in general. Highlighting can be based on search (similar to Logs UI
  
  Also provide condition like bytes>1000

cc: @kertal

[flash1293]

Related to

Evaluate if the side menu can be similar to Lens, provide consistency and the ability to see field context, except for the dragging of fields it can share the same capabilities

#41108

[elasticmachine]

Pinging @elastic/kibana-design

[kertal]

If our valued @elastic/kibana-design team is ok with aligning the side menu with Lens (which I think is a good idea) this would be my next step in the deangularization + Euification of Discover

[ryankeairns]

@mdefazio this meta issue tracks the Discover enhancements we discussed previously.

@kertal would a mockup be helpful for the Lens-style side menu?

[kertal]

@ryankeairns if we agree doing it this way, definitely. mocks would be very welcome 👍

[cchaos]

@kertal I'm guessing you talked with @AlonaNadler and @wylieconlon about reusing the Lens component for this? How would you integrate the work that's happening on the Lens side? Would you collaborate on the effort? How much feature parity will there be when Lens is still Beta?

I do like the idea of sharing this component, by the way.

[kertal]

@cchaos That's the things I'd start to evaluate now, if you're saying, it would be a good approach going into that direction from design perspective. maybe we should have a session about this?

[kertal]

An alternative approach that comes to my mind would be to move the field selection into the new grid, but then the new grids field selector would need more features like searching in long field lists, displaying details about data etc.

[ryankeairns]

Seems like quick session would be a good start as we have people coming at this from a couple of angles. Thanks @kertal !

[AlonaNadler]

An alternative approach that comes to my mind would be to move the field selection into the new grid

I think we will still need the left side menu listing the fields. Removing it will be too big of a change and users look at discover not only using the data grid. We can make it collapsable if we want which will give more screen real estate when using grid

@wylieconlon is currently working on the context pop up for each field in Lens, the approach is different than the one Discover uses, it is worth evaluating it in the context of discover.
Currently when selecting a field in Discover, it does top values. Wylie's PR is closer to ML data visualizer were based on the field type it provide a different context to the field value. We should check that it doesn't have any performance drawback

[mdefazio]

Here's a first go at updating the left side menu based on our the direction here

Prototype

Quick gif for reference:

The prototype doesn't show it properly, but we are proposing using an 'add/remove' icon as opposed to the full text on hover to add it to the selected fields

[AlonaNadler]

Thanks @mdefazio looks excellent, visually it is an improvement, consistent with Lens, distinguishing between fields is easy and good, search and filter by type is a great improvement.
Would be good if we can adjust the width to support long fields currently discover truncates the fields name

Maybe a route similar to how it is done currently in visualize left side configurator, allowing users to increase the width

[kertal]

looking forward to implement it @mdefazio
@AlonaNadler the adjustment of the columns is very easy to overlook currently in visualize, I'll take a look at the underlying code

[mdefazio]

Here is a first go at how the DataGrid could look inside Discover. I'm showing a popover (on click) for one cell. I'm also showing how the column renaming could work. Let me know your thoughts.
@kertal @AlonaNadler @timroes

[AlonaNadler]

Wow, this is so exciting to see !!! ship it. :)

Few suggestions:

• when clicking on a field can we allow users to click on the cell itself instead of trying to get to their mouse on the expand icon
• instead of the magnifying glass icon which is not clear to many of our users can we have descriptive options, e.g. filter on the value, exclude value
• sorting on a single column is what most users do, can we do that on the column itself
• if using multiple fields sorting can the sorting label show how many fields are sorted, similar to the filter fields counter

[snide]

sorting on a single column is what most users do, can we do that on the column itself

This will be in a later release of the data grid. All the elements are in it to make this available, and my guess is it would likely be implemented as a feature in data grid component before the discover project is shipped. You can stop requesting it in every channel and making me guilty! 😆

if using multiple fields sorting can the sorting label show how many fields are sorted, similar to the filter fields counter

It already does this. 🎉

when clicking on a field can we allow users to click on the cell itself instead of trying to get to their mouse on the expand icon

I'll need to think about how to handle this. In the majority of cases data grid cells won't have actions, so it would be annoying to always pop up the expansion. We could probably do something like this with a special prop. Likely a soon after V1 for the data grid component, that discover would be able to implement before they ship.

instead of the magnifying glass icon which is not clear to many of our users can we have descriptive options

👍 to this.

[mdefazio]

Updated popover with descriptions next to the magnifying glass

[ryankeairns]

@mdefazio yeesssss, I love that! Much clearer, nice design.

[AlonaNadler]

Nice !
Can we make the entire cell clickable, I see users challenging in hitting the icon with their mouse in Lens when we have icon in a similar location and this operation will be done quite frequently

[ryankeairns]

What Michael has mocked up here is how the datagrid currently works, so any change would come on the EUI side. The first version of datagrid is nearing release, so let's see that through, get a feel for things, and follow up with any enhancements requests (in EUI).

[timroes]

#40719 Has another comment about real estate in the discover app, we might want to take into consideration during the redesign process.

[timroes]

Closing this in favor of #92573