Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for templating in configs #6251

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Add support for templating in configs #6251

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
1e4e7a2
Add support for templating in configs.
robholland Jul 8, 2024
aefdebd
Adjust missed test.
robholland Jul 8, 2024
5293fc7
Add a port of the docker config template to sprig.
robholland Aug 12, 2024
1efd952
Improve error messages.
robholland Aug 13, 2024
2e95e8b
Merge branch 'main' into rh-config-template
robholland Sep 25, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 18 additions & 1 deletion common/config/loader.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,13 @@
package config

import (
"bytes"
"fmt"
stdlog "log"
"os"
"text/template"

"github.com/Masterminds/sprig/v3"
"gopkg.in/validator.v2"
"gopkg.in/yaml.v3"
)
Expand Down Expand Up @@ -90,14 +93,28 @@ func Load(env string, configDir string, zone string, config interface{}) error {
// TODO: remove log dependency.
stdlog.Printf("Loading config files=%v\n", files)

templateFuncs := sprig.FuncMap()

for _, f := range files {
// This is tagged nosec because the file names being read are for config files that are not user supplied
// #nosec
data, err := os.ReadFile(f)
if err != nil {
return err
}
err = yaml.Unmarshal(data, config)

tpl, err := template.New("config").Funcs(templateFuncs).Parse(string(data))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this compatible with the existing config_template.yaml? it looks like sprig provides a function used like env "VAR" but the template is written with things like .Env.VAR. do we have to migrate the template?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The template would need adjusting. Sprig's default argument ordering is reversed from the dockerize version. There are probably a few other gotchas. I would suggest leaving the existing template as it is and creating a new template under config/docker.yaml with the updated syntax so that containers using dockerize will continue to work until we deprecate that. Newer containers can use this new functionality with the updated config file. I'll add a working config file to this PR.

if err != nil {
return fmt.Errorf("template parsing error: %w", err)
}

var rendered bytes.Buffer
err = tpl.Execute(&rendered, nil)
if err != nil {
return fmt.Errorf("template execution error: %w", err)
}

err = yaml.Unmarshal(rendered.Bytes(), config)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It also would be very useful for debugging purpose to check what is in rendered. And not only in case of template error. Template syntax might be fine, but produce different output. Write it to stdout? I am ok, if you do it in separate PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed to put this into a tdbg command for now, and later move into the server binary once we're happy (due to backwards compat requirements).

if err != nil {
return err
}
Expand Down
22 changes: 11 additions & 11 deletions common/config/loader_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ func (s *LoaderSuite) TestBaseYaml() {
var cfg testConfig
err = Load(env, dir, zone, &cfg)
s.Nil(err)
s.Equal("hello__", cfg.Items.Item1)
s.Equal("HELLO__", cfg.Items.Item1)
s.Equal("world__", cfg.Items.Item2)
}
}
Expand All @@ -94,15 +94,15 @@ func (s *LoaderSuite) TestHierarchy() {
item1 string
item2 string
}{
{"", "", "hello_development_", "world_development_"},
{"", "dca", "hello_development_", "world_development_"},
{"", "pdx", "hello_development_", "world_development_"},
{"development", "", "hello_development_", "world_development_"},
{"development", "dca", "hello_development_", "world_development_"},
{"development", "pdx", "hello_development_", "world_development_"},
{"prod", "", "hello_prod_", "world_prod_"},
{"prod", "dca", "hello_prod_dca", "world_prod_dca"},
{"prod", "pdx", "hello_prod_", "world_prod_"},
{"", "", "HELLO_DEVELOPMENT_", "world_development_"},
{"", "dca", "HELLO_DEVELOPMENT_", "world_development_"},
{"", "pdx", "HELLO_DEVELOPMENT_", "world_development_"},
{"development", "", "HELLO_DEVELOPMENT_", "world_development_"},
{"development", "dca", "HELLO_DEVELOPMENT_", "world_development_"},
{"development", "pdx", "HELLO_DEVELOPMENT_", "world_development_"},
{"prod", "", "HELLO_PROD_", "world_prod_"},
{"prod", "dca", "HELLO_PROD_DCA", "world_prod_dca"},
{"prod", "pdx", "HELLO_PROD_", "world_prod_"},
}

for _, tc := range testCases {
Expand Down Expand Up @@ -130,6 +130,6 @@ func buildConfig(env, zone string) string {
item2 := concat("world", concat(env, zone))
return `
items:
item1: ` + item1 + `
item1: {{ "` + item1 + `" | upper }}
item2: ` + item2
}
Loading
Loading