[elasticsearch] use security by default

[jmlrt]

This commit update Elasticsearch chart to use security by default.

• Adds a new Secret template for Elasticsearch password with a
  randomized password if secret.password isn't defined.

• Adds instructions to retrieve the password in Elasticsearch chart
  deployment notes.

• Also, remove usage of ELASTIC_USERNAME variable because it
  don't seem to be supported anymore by Elasticsearch

The other charts will be updated in follow-up PRs to use the proper
credentials

Relates to #1375

[jmlrt]

@mark-vieira @jkakavas, FYI, this PR is addressing #1375 by handling credentials generation at Helm chart level instead of Elasticsearch level when no credentials are provided.

This way we can have the credentials in a K8S secret then use it as an environment variable in the Elasticsearch pod for the health check script.

The other Elastic charts will also need to be updated in follow-up PRs to be able to use this new secret when they are using some Elasticsearch output.

[mark-vieira]

Apologies my helm-foo is pretty weak. So we're not requiring folks to either explicitly enable or disable security, yes? And if enabled, they must also provide a username/password to get passed into the chart, or, provide an existing k8s secret from which to get this configuration?

[framsouza]

I tested it locally and everything worked as expected, I just put a comment on the ELASTIC_USER handling, we may consider handle it as a normal variable instead of secret as the user name doesn't seems to be a valid secret

[framsouza]

I think here we can set the username instead of use pointing to the secret, as by default the user name is "elastic" I don't think it's a fit to be handle as a secret key.

  - name: ELASTIC_USERNAME
     value: elastic

What do you think?

[framsouza]

Same here, I would suggest to not refer to the secret for the username:

I think here we can set the username instead of use pointing to the secret, as by default the user name is "elastic" I don't think it's a fit to be handle as a secret key.

  - name: ELASTIC_USERNAME
     value: elastic

[framsouza]

Same here

[jmlrt]

Apologies my helm-foo is pretty weak. So we're not requiring folks to either explicitly enable or disable security, yes? And if enabled, they must also provide a username/password to get passed into the chart, or, provide an existing k8s secret from which to get this configuration?

In the default config, we don't explicitly configure xpack.security.enabled as we are using the default elasticsearch.yml file provided by the Elasticsearch Docker image.

Passing an existing K8S secret created outside of the chart was already an option previously to define ELASTIC_USERNAME and ELASTIC_PASSWORD environment variables.

With this PR it is still possible to use an external secret but with the default behavior the Helm chart will create the secret and handle it as part of the chart using a password generated by Helm itself or defined by overriding secret.password value.

• Helm install with default config:

$ helm install es ./elasticsearch
NAME: es
LAST DEPLOYED: Tue Oct 12 10:25:08 2021
NAMESPACE: jmlrt-test
STATUS: deployed
REVISION: 1
NOTES:
1. Watch all cluster members come up.
  $ kubectl get pods --namespace=jmlrt-test -l app=elasticsearch-master -w
2. Retrieve credentials.
  $ kubectl get secrets --namespace=jmlrt-test elasticsearch-master-credentials -ojsonpath='{.data.username}' | base64 -d
  $ kubectl get secrets --namespace=jmlrt-test elasticsearch-master-credentials -ojsonpath='{.data.password}' | base64 -d
3. Test cluster health using Helm test.
  $ helm --namespace=jmlrt-test test es

$ kubectl get secrets --namespace=jmlrt-test elasticsearch-master-credentials -ojsonpath='{.data.username}' | base64 -d
elastic%

$ kubectl get secrets --namespace=jmlrt-test elasticsearch-master-credentials -ojsonpath='{.data.password}' | base64 -d
CZbiUV9y6wXgcrIc%

• Helm install with secret.password defined:

$ helm install es ./elasticsearch --set secret.password=myelasticsecret
NAME: es
LAST DEPLOYED: Tue Oct 12 10:32:55 2021
NAMESPACE: jmlrt-test
STATUS: deployed
REVISION: 1
NOTES:
1. Watch all cluster members come up.
  $ kubectl get pods --namespace=jmlrt-test -l app=elasticsearch-master -w
2. Retrieve credentials.
  $ kubectl get secrets --namespace=jmlrt-test elasticsearch-master-credentials -ojsonpath='{.data.username}' | base64 -d
  $ kubectl get secrets --namespace=jmlrt-test elasticsearch-master-credentials -ojsonpath='{.data.password}' | base64 -d
3. Test cluster health using Helm test.
  $ helm --namespace=jmlrt-test test es

$ kubectl get secrets --namespace=jmlrt-test elasticsearch-master-credentials -ojsonpath='{.data.username}' | base64 -d
elastic%

$ kubectl get secrets --namespace=jmlrt-test elasticsearch-master-credentials -ojsonpath='{.data.password}' | base64 -d
myelasticsecret%

[jmlrt]

I tested it locally and everything worked as expected, I just put a comment on the ELASTIC_USER handling, we may consider handle it as a normal variable instead of secret as the user name doesn't seems to be a valid secret

Indeed, changing the username via the ELASTIC_USERNAME variable doesn't seem to be supported by Elasticsearch anymore, so it's better to hardcode elastic everywhere.

@framsouza PTAL 🙏🏻

[framsouza]

looks great now ;)

[jmlrt]

Merging as Elasticsearch chart tests are successful and the other charts will be fixed in follow-up PRs.