Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[confmap] Use original string value on expansion #10603

Closed
wants to merge 3 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions .chloggen/mx-psi_as-string.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Use this changelog template to create an entry for release notes.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: bug_fix

# The name of the component, or a single word describing the area of concern, (e.g. otlpreceiver)
component: confmap

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: When passing value to a string field through the `${<provider>:URI}` or `${URI}` syntax, the original raw string value is now preserved.

# One or more tracking issues or pull requests related to the change
issues: [10603]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext: |
- This change only applies under the `confmap.unifyEnvVarExpansion` feature gate.

# Optional: The change log or logs in which this entry should be included.
# e.g. '[user]' or '[user, api]'
# Include 'user' if the change is relevant to end users.
# Include 'api' if there is a change to a library API.
# Default: '[user]'
change_logs: []
23 changes: 9 additions & 14 deletions confmap/internal/e2e/types_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -98,27 +98,22 @@ func TestTypeCasting(t *testing.T) {
{
value: "\"0123\"",
targetField: TargetFieldString,
expected: "0123",
},
{
value: "\"0123\"",
targetField: TargetFieldInt,
expected: 83,
expected: "\"0123\"",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This breaks users right?
If I entered endpoint: "http://localhost:4317" instead of my exporter using http://localhost:4317 would it attempt to use a value that contains double-quote literals?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I entered endpoint: "http://localhost:4317" instead of my exporter using http://localhost:4317 would it attempt to use a value that contains double-quote literals?

That would happen if you did

endpoint: ${env:ENDPOINT}

with ENDPOINT set to "http://localhost:4317" (quotes included). The changes only affects how the values passed via a provider are parsed

},
{
value: "\"0123\"",
targetField: TargetFieldInlineString,
expected: "inline field with 0123 expansion",
expected: "inline field with \"0123\" expansion",
},
{
value: "!!str 0123",
targetField: TargetFieldString,
expected: "0123",
expected: "!!str 0123",
codeboten marked this conversation as resolved.
Show resolved Hide resolved
},
{
value: "!!str 0123",
targetField: TargetFieldInlineString,
expected: "inline field with 0123 expansion",
expected: "inline field with !!str 0123 expansion",
},
{
value: "t",
Expand Down Expand Up @@ -228,27 +223,27 @@ func TestStrictTypeCasting(t *testing.T) {
{
value: "\"0123\"",
targetField: TargetFieldString,
expected: "0123",
expected: "\"0123\"",
},
{
value: "\"0123\"",
targetField: TargetFieldInt,
unmarshalErr: "'field' expected type 'int', got unconvertible type 'string', value: '0123'",
unmarshalErr: "'field' expected type 'int', got unconvertible type 'string', value: '\"0123\"'",
},
{
value: "\"0123\"",
targetField: TargetFieldInlineString,
expected: "inline field with 0123 expansion",
expected: "inline field with \"0123\" expansion",
},
{
value: "!!str 0123",
targetField: TargetFieldString,
expected: "0123",
expected: "!!str 0123",
},
{
value: "!!str 0123",
targetField: TargetFieldInlineString,
expected: "inline field with 0123 expansion",
expected: "inline field with !!str 0123 expansion",
},
{
value: "t",
Expand Down
11 changes: 9 additions & 2 deletions confmap/provider.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ import (

"go.uber.org/zap"
"gopkg.in/yaml.v3"

"go.opentelemetry.io/collector/internal/featuregates"
)

// ProviderSettings are the settings to initialize a Provider.
Expand Down Expand Up @@ -138,9 +140,14 @@ func NewRetrievedFromYAML(yamlBytes []byte, opts ...RetrievedOption) (*Retrieved
return nil, err
}

switch v := rawConf.(type) {
switch rawConf.(type) {
case string:
opts = append(opts, withStringRepresentation(v))
if featuregates.UseUnifiedEnvVarExpansionRules.IsEnabled() {
// If rawConf is a string, we override it with the raw yamlBytes.
// This is the original ${ENV} expansion behavior.
rawConf = string(yamlBytes)
}
opts = append(opts, withStringRepresentation(string(yamlBytes)))
Copy link
Member Author

@mx-psi mx-psi Jul 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is under the strict types gate

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This changes the behavior when the feature gate is disabled. Before it would set the string representation to be the yaml parsed version of the string (so \n is removed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, but the string representation is only used under the strict types feature gate, so it's also under a feature gate

case int, int32, int64, float32, float64, bool:
opts = append(opts, withStringRepresentation(string(yamlBytes)))
}
Expand Down
4 changes: 2 additions & 2 deletions confmap/provider_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -85,8 +85,8 @@ func TestNewRetrievedFromYAMLString(t *testing.T) {
},
{
yaml: "\"string\"",
value: "string",
altStrRepr: "string",
value: "\"string\"",
altStrRepr: "\"string\"",
},
{
yaml: "123",
Expand Down
4 changes: 2 additions & 2 deletions docs/rfcs/env-vars.md
Original file line number Diff line number Diff line change
Expand Up @@ -216,7 +216,7 @@ loading a field with the braces syntax, `env` syntax.
| `0123` | integer | 83 | 83 | 83 | n/a |
| `0123` | string | 0123 | 83 | 0123 | 0123 |
| `0xdeadbeef` | string | 0xdeadbeef | 3735928559 | 0xdeadbeef | 0xdeadbeef |
| `"0123"` | string | "0123" | 0123 | 0123 | 0123 |
| `!!str 0123` | string | !!str 0123 | 0123 | 0123 | 0123 |
| `"0123"` | string | "0123" | 0123 | "0123" | "0123" |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add a row for foo\nbar

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add foo::

| `!!str 0123` | string | !!str 0123 | 0123 | !!str 0123 | !!str 0123 |
| `t` | boolean | true | true | Error: mapping string to bool | n/a |
| `23` | boolean | true | true | Error: mapping integer to bool | n/a |
Loading