Clustering Kibana

[kobelb]

Currently, there isn't an easy way to create a cluster of Kibana processes that all run on a single machine. It's common for us to recommend that users do so manually and put all of the Kibana instances behind a load balancer. However, this is a rather large burden to place on our users and makes it difficult to take advantage of a multi-processor machines because Node.js is largely single-threaded, outside of the libuv worker threads.

Node.js has a Cluster module, which can be used to have multiple Kibana HTTP servers listening on a single port. It seems worthwhile to explore whether or not this is a viable option for Kibana to take advantage of to address this pain point.

Some interesting behaviors we'd need to take into consideration (I'm sure there are many others):

• Kibana proesses currently require their own data folders to write their UUID, we'd need to figure out how to share this folder
• The logging.dest setting specifies a file where Kibana should output its logs. Can we have all processes of Kibana writing to the same file? If so, it feels like we should designate which log entry is coming from which instance.
• How will metricbeat collect stats from all instances? If we use the Cluster module, it'll just pick one instance to and use that for /api/stats.
• What will happen if all processes try to run saved-object migrations at the same time? Theoretically, it seems like this will be fine, but it might make the startup slow.
• Should all processes run taskManager? There are concurrency controls that are either currently or will be put in place to ensure that Reporting only spawns a single instance of Chromium at a time, since Chromium consumes a lot of resources.

[elasticmachine]

Pinging @elastic/kibana-platform (Team:Platform)

[kobelb]

Dupe of #68626, closing