[Discuss] Use soClient instead of esClient in usage collector fetch

[sulemanof]

Here is the initial suggestion to use soClient instead of esClient to fetch saved objects from .kibana index. Here is the concern:

AFAIK, ES is planning to forbid direct access to the system indices in the future

Question:
Do we need to do the same replacement for other places, where we use esClient for searching in .kibana index?

Couple of places I found in OSS, but maybe this is not the full list:

• src/plugins/data/server/search/collectors/fetch.ts;
• src/plugins/home/server/services/sample_data/usage/collector.ts;
• src/plugins/vis_type_vega/server/usage_collector/get_usage_collector.ts;
• src/plugins/visualizations/server/usage_collector;

[sulemanof]

cc @afharo

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[pgayvallet]

Core is going to expose a custom ES client to allow interacting with system indices: #82716

That being said, when querying for SOs, if you can use the SO client instead of the 'raw' ES client, then you probably should.

Most significant exception being if you need to perform aggregations (well, until questions around #64002 are addressed)

[mshustov]

@pgayvallet Can be closed as an idea with adding another ES client is rejected? Kibana plugins continue having access to the system indices.

[pgayvallet]

Can be closed as an idea with adding another ES client is rejected

There is no longer any hard requirement to address the issue.

But the

When querying for SOs, if you can use the SO client instead of the 'raw' ES client, then you probably should.

Part remain valid. If the collectors are able to retrieve what they want using the SO client, they should. But this would be just a cleanup/enhancement