-
Notifications
You must be signed in to change notification settings - Fork 8.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
All requests to Kibana server have Connection: keep-alive
#44537
Comments
Pinging @elastic/kibana-platform |
Nope. That was an unintentional change. Before #39047 we had 2 hapi server instances and created them in the next sequence:
After #39047:
What is the main problem if we follow the standard behavior and set |
Personally, it makes sense to maintain persistent connections to avoid the extra network latency of handshake, but I'm not really suggesting we should do one or the other. I'm just trying to understand how to communicate this change to stack monitoring users who saw a huge jump in concurrent connections since 7.3. It sounds like the current thinking is we should maintain persistent connections which means we might need to re-think how we show this metric to users, since it seems fairly unhelpful right now. I'm not sure if there needs to be a more widespread discussion about this change, but if so, can we keep it here on this ticket, or at least link the new issue from here so we can all follow? |
Is it unhelpful? As far as I understand the monitoring is correct. There really are 100s of open connections. The question to me is, is this many open connections a problem for Kibana? Node.js is exceptionally good at handling thousands of open connections with very little overhead, so on it's face I don't think we have any data that there's an actual technical problem yet. This seems to be an expectation setting problem rather than a technical one. I do we think we should consider adding the |
++. I tried to say this in my recent comment, but I might not have been clear, but yes I agree!
Based on the above, I don't think this is the necessarily the question raised for this. Maybe we need two separate tickets, but I made this ticket to understand the reason for the change and help users understand why the number is higher than before without them doing anything differently. The only thing left to resolve on my front is if this change is expected to stay or not - I'm assuming the answer is yes but I'd love to verify this and then we can go about how to handle that on the stack monitoring end. |
Any update on this from the platform team? Mainly:
|
I haven't had time to think about this deeply, but I don't see any reason for us to go back to Maybe keep-alive is also preferred for healthchecks so load balancers don't keep opening and closing connections unnecessarily. I do regret that this change happened without us expecting it. Are we seeing a lot of feedback in the field that this is confusing Monitoring users? |
Not a lot, no. I've only heard a single report, but I have to imagine anyone relying on these metrics will notice the change. Any thoughts on how we can communicate this to users @cachedout? |
TBH, I doubt very much that this is going to be noticed by more than a very small handful of people. My recommendation is that we see if we can get them into the release notes, so if a person does indeed notice that they'll hopefully get that page returned in search results -- assuming that they don't find this GH issue first. @lcawl Perhaps you can help here. Do we ever retroactively modify release notes when we realize that we forgot to mention something that could have been helpful? If so, can you help us navigate the process of doing so? |
This is affecting 7.3 and later versions of Kibana.
Prior to 7.3, this is what all requests look like:
Starting in 7.3, this is what all requests look like:
Notice the value of the
connection
header.Kudos to @tylersmalley and @jbudz for helping identify the offensive PR: #39047
I'm not sure if this change was intentional or not, but one of the issues with this change is the number of concurrent connections reported for Kibana monitoring data is way too high as these connections are held for some period of time (In my testing, it's 2min).
Maybe we need this change moving forward, but I can't find any information on the why to understand how to communicate this change in behavior for stack monitoring users.
cc @restrry @joshdover
The text was updated successfully, but these errors were encountered: