Error logs when clients connect over HTTP instead of HTTPS

[jportner]

Kibana version:

7.9 (tested, this likely affects many older versions)

Describe the bug:

When Kibana has HTTPS enabled, but a client tries to connect over HTTP, the client gets redirected to HTTPS as expected an empty response as expected*. However, ugly error messages show up in the logs:

server   error  [13:44:35.450] [error][client][connection] Error: 4388156864:error:1408F09C:SSL routines:ssl3_get_record:http request:../deps/openssl/openssl/ssl/record/ssl3_record.c:322:

We have code that is supposed to downgrade these error messages to the debug log level:

kibana/src/legacy/server/logging/log_interceptor.js

Lines 126 to 128 in 003fcb1

	downgradeIfHTTPWhenHTTPS(event) {
	return downgradeIfErrorMessage(GET_CLIENT_HELLO, event);
	}

However, that code is broken. It was originally introduced in Kibana 6.0 (#11209), back when we were using Node 6.10.2 and Hapi 14.2.0. Now, we are using Node 10.22.0 (which includes bundled OpenSSL 1.1.1) and Hapi 17.6.0.

Steps to reproduce:

1. Start Kibana with SSL enabled (server.ssl.enabled: true)
2. Connect to Kibana using a client via HTTP
3. Observe the aforementioned error messages in the Kibana logs

Expected behavior:

These error messages should be downgraded to the debug log level. The error message has simply changed at some point since this log interceptor was originally added.

Any additional context:

I added some additional logging to the log interceptor to verify:

index 07545570445..644f18afba5 100644
--- a/src/legacy/server/logging/log_interceptor.js
+++ b/src/legacy/server/logging/log_interceptor.js
@@ -58,6 +58,15 @@ function downgradeIfErrorMessage(match, event) {
   const errorMessage = get(event, 'error.message');
   const matchesErrorMessage = isClientError && doesMessageMatch(errorMessage, match);
 
+  if (isClientError && !matchesErrorMessage && match === GET_CLIENT_HELLO) {
+    const { error } = event;
+    console.log(
+      `Not downgraded (match ${match.toString()}): ${JSON.stringify(
+        error,
+        Object.getOwnPropertyNames(error)
+      )}`
+    );
+  }
   if (!matchesErrorMessage) return null;
 
   return {

Resulting logs show this:

Not downgraded (match /GET_CLIENT_HELLO:http/): {"stack":"Error: 4584650176:error:1408F09C:SSL routines:ssl3_get_record:http request:../deps/openssl/openssl/ssl/record/ssl3_record.c:322:\n","message":"4584650176:error:1408F09C:SSL routines:ssl3_get_record:http request:../deps/openssl/openssl/ssl/record/ssl3_record.c:322:\n"}
server   error  [14:52:58.712] [error][client][connection] Error: 4584650176:error:1408F09C:SSL routines:ssl3_get_record:http request:../deps/openssl/openssl/ssl/record/ssl3_record.c:322:

So, it looks like this is an easy fix, but unfortunately we'll still have to resort to error message matching.

---

Note: tagging Platform as they are technically the code owner, but also Security because I'm about to submit a PR to fix it 🙂

*Update: redirect does not happen / has nothing to do with it, that was just my browser being helpful.

[elasticmachine]

Pinging @elastic/kibana-platform (Team:Platform)

[elasticmachine]

Pinging @elastic/kibana-security (Team:Security)