Allow yaml values for dynamic node settings #85186
Conversation
Environment variables are allowed as substitutions within elasticsearch.yml. Additionally, command line settings are added into the parsed settings. However, both of these are raw strings applied after the node yml file has been parsed. This commit moves environment substitution to occur before parsing elasticsearch.yml, and override processing to happen with a separate yaml parser so that both can allow yaml parsing. Note that environment substitution is not the only type of substitution (there is also setting substitution, which needs parsed setting keys), so the existing replacement mechanism is not touched here except to remove environment handling. closes elastic#65577
rjernst
added
>enhancement
:Core/Infra/Core
|
Pinging @elastic/es-core-infra (Team:Core/Infra) |
|
Hi @rjernst, I've created a changelog YAML for you. |
| } | ||
| var is = new ByteArrayInputStream(builder.toString().getBytes(StandardCharsets.UTF_8)); | ||
| // fake the resource name so it loads yaml | ||
| output.loadFromStream("overrides.yml", is, false); |
There was a problem hiding this comment.
I'm confused why this is necessary. Doesn't the later call to initializeSettings take care of replacing any remaining values? How many "passes" are necessary?
There was a problem hiding this comment.
Oops, I meant to remove the putProperties in initializeSettings (I've done so now). The idea is that instead of putting the values directly (which may be json), we load them here as if they set in a yaml file.
Given that this has tests, I'm going to go ahead and merge, and I will followup with a higher level integration test. I'd like to avoid package tests because none of this is different across platforms. |
Not across platforms, but environment variable handling is special for Docker packaging. |
|
rjernst was this fix for 8.4.0 as well ? |
Environment variables are allowed as substitutions within
elasticsearch.yml. Additionally, command line settings are added into
the parsed settings. However, both of these are raw strings applied
after the node yml file has been parsed.
This commit moves environment substitution to occur before parsing
elasticsearch.yml, and override processing to happen with a separate
yaml parser so that both can allow yaml parsing. Note that environment
substitution is not the only type of substitution (there is also setting
substitution, which needs parsed setting keys), so the existing
replacement mechanism is not touched here except to remove environment
handling.
closes #65577