Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use dynamic configuration model in bundles #1098

Merged
merged 142 commits into from
Feb 16, 2024
Merged

Use dynamic configuration model in bundles #1098

merged 142 commits into from
Feb 16, 2024

Commits on Oct 31, 2023

  1. Work on convert.FromTyped

    pietern committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    ab894fa View commit details
    Browse the repository at this point in the history
  2. More work on ToTyped

    pietern committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    d23e201 View commit details
    Browse the repository at this point in the history
  3. More tests

    pietern committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    36ea807 View commit details
    Browse the repository at this point in the history
  4. Cleanup

    pietern committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    b9191a4 View commit details
    Browse the repository at this point in the history
  5. wip

    pietern committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    7968bcb View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    9b1ed86 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    effb185 View commit details
    Browse the repository at this point in the history

Commits on Nov 1, 2023

  1. tmp

    pietern committed Nov 1, 2023
    Configuration menu
    Copy the full SHA
    314ab30 View commit details
    Browse the repository at this point in the history

Commits on Nov 15, 2023

  1. Configuration menu
    Copy the full SHA
    d9d71c3 View commit details
    Browse the repository at this point in the history
  2. Rename variable bundle -> b

    pietern committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    bacac76 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    33f5e75 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    e7091f5 View commit details
    Browse the repository at this point in the history
  5. Tmp

    pietern committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    bf54271 View commit details
    Browse the repository at this point in the history

Commits on Nov 24, 2023

  1. Configuration menu
    Copy the full SHA
    a68f134 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    a93b508 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    1dad855 View commit details
    Browse the repository at this point in the history
  4. Fix

    pietern committed Nov 24, 2023
    Configuration menu
    Copy the full SHA
    41658d4 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    c8f02dc View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    0d2d4b8 View commit details
    Browse the repository at this point in the history

Commits on Dec 18, 2023

  1. Configuration menu
    Copy the full SHA
    ca5d5d4 View commit details
    Browse the repository at this point in the history
  2. Add config walker

    pietern committed Dec 18, 2023
    Configuration menu
    Copy the full SHA
    f18ba50 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    f7b3290 View commit details
    Browse the repository at this point in the history
  4. Mutator tests pass

    pietern committed Dec 18, 2023
    Configuration menu
    Copy the full SHA
    21006e5 View commit details
    Browse the repository at this point in the history

Commits on Dec 20, 2023

  1. Construct path from string

    pietern committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    97908b6 View commit details
    Browse the repository at this point in the history
  2. wip

    pietern committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    4543d2d View commit details
    Browse the repository at this point in the history
  3. Functionality to walk a config.Value tree

    This change adds:
    * A `config.Walk` function to walk a configuration tree
    * A `config.Path` type to represent a value's path inside a tree
    * Functions to create a `config.Path` from a string, or convert one to a string
    pietern committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    875454d View commit details
    Browse the repository at this point in the history
  4. Comments

    pietern committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    89cae7c View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    5695e03 View commit details
    Browse the repository at this point in the history
  6. wip

    pietern committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    ad3b62b View commit details
    Browse the repository at this point in the history
  7. wip

    pietern committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    4347007 View commit details
    Browse the repository at this point in the history
  8. Fix job with spark conf test

    pietern committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    bd64d1d View commit details
    Browse the repository at this point in the history

Commits on Dec 21, 2023

  1. Configuration menu
    Copy the full SHA
    962fa96 View commit details
    Browse the repository at this point in the history

Commits on Dec 22, 2023

  1. Address comments

    pietern committed Dec 22, 2023
    Configuration menu
    Copy the full SHA
    40e1425 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    9ee9233 View commit details
    Browse the repository at this point in the history
  3. Rename libs/config -> libs/dyn

    The name "dynamic value", or "dyn" for short, is more descriptive than the
    opaque "config". Also, it conveniently does not alias with other packages in
    the repository, or (popular ones) elsewhere.
    pietern committed Dec 22, 2023
    Configuration menu
    Copy the full SHA
    fb7ec6b View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    b71d8d4 View commit details
    Browse the repository at this point in the history
  5. Finalize package rename

    pietern committed Dec 22, 2023
    Configuration menu
    Copy the full SHA
    8fc3ab5 View commit details
    Browse the repository at this point in the history
  6. More renaming

    pietern committed Dec 22, 2023
    Configuration menu
    Copy the full SHA
    3dbadb4 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    8a11cee View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    69370c4 View commit details
    Browse the repository at this point in the history
  9. Use resource key as name in permissions code

    The code relied on the `Name` property being accessible for every resource.
    This is generally true, but because these property structs are embedded as
    pointer, they can be nil. This is also why the tests had to initialize the
    embedded struct to pass. This changes the approach to use the keys from the
    resource map instead, so that we no longer rely on the non-nil embedded struct.
    
    Note: we should evaluate whether we should turn these into values instead of
    pointers. I don't recall if we get value from them being pointers.
    pietern committed Dec 22, 2023
    Configuration menu
    Copy the full SHA
    cdbea58 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    21cf860 View commit details
    Browse the repository at this point in the history

Commits on Dec 27, 2023

  1. Configuration menu
    Copy the full SHA
    643ae8d View commit details
    Browse the repository at this point in the history

Commits on Jan 3, 2024

  1. Configuration menu
    Copy the full SHA
    892b344 View commit details
    Browse the repository at this point in the history

Commits on Jan 4, 2024

  1. wip git

    pietern committed Jan 4, 2024
    Configuration menu
    Copy the full SHA
    68135cb View commit details
    Browse the repository at this point in the history
  2. More dyn work

    pietern committed Jan 4, 2024
    Configuration menu
    Copy the full SHA
    b50cc8a View commit details
    Browse the repository at this point in the history
  3. remove nop

    pietern committed Jan 4, 2024
    Configuration menu
    Copy the full SHA
    2f36c7e View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    aa19b48 View commit details
    Browse the repository at this point in the history
  5. Inline

    pietern committed Jan 4, 2024
    Configuration menu
    Copy the full SHA
    0213861 View commit details
    Browse the repository at this point in the history
  6. Define dyn.InvalidValue

    pietern committed Jan 4, 2024
    Configuration menu
    Copy the full SHA
    8f3cd2e View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    4470722 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    1cf75ee View commit details
    Browse the repository at this point in the history
  9. dyn wip

    pietern committed Jan 4, 2024
    Configuration menu
    Copy the full SHA
    83fb21c View commit details
    Browse the repository at this point in the history
  10. More dyn work

    pietern committed Jan 4, 2024
    Configuration menu
    Copy the full SHA
    d80a8ce View commit details
    Browse the repository at this point in the history
  11. More dyn work

    pietern committed Jan 4, 2024
    Configuration menu
    Copy the full SHA
    b825ccd View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    fab40b1 View commit details
    Browse the repository at this point in the history

Commits on Jan 10, 2024

  1. wip

    pietern committed Jan 10, 2024
    Configuration menu
    Copy the full SHA
    2ab8065 View commit details
    Browse the repository at this point in the history
  2. comment

    pietern committed Jan 10, 2024
    Configuration menu
    Copy the full SHA
    47eb229 View commit details
    Browse the repository at this point in the history

Commits on Jan 12, 2024

  1. Configuration menu
    Copy the full SHA
    c9330a4 View commit details
    Browse the repository at this point in the history

Commits on Jan 17, 2024

  1. dynvar

    pietern committed Jan 17, 2024
    Configuration menu
    Copy the full SHA
    b3113d6 View commit details
    Browse the repository at this point in the history

Commits on Jan 19, 2024

  1. More dynvar

    pietern committed Jan 19, 2024
    Configuration menu
    Copy the full SHA
    335d3de View commit details
    Browse the repository at this point in the history

Commits on Jan 23, 2024

  1. Comment

    pietern committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    b9ea101 View commit details
    Browse the repository at this point in the history
  2. Make variable tests pass

    pietern committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    9000bcb View commit details
    Browse the repository at this point in the history
  3. .

    pietern committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    ab42361 View commit details
    Browse the repository at this point in the history

Commits on Jan 24, 2024

  1. Configuration menu
    Copy the full SHA
    a95be0f View commit details
    Browse the repository at this point in the history
  2. Add functionality to visit values in dyn.Value tree

    This change adds the following functions:
    * `dyn.Get(value, "foo.bar") -> (dyn.Value, error)`
    * `dyn.Set(value, "foo.bar", newValue) -> (dyn.Value, error)`
    * `dyn.Map(value, "foo.bar", func) -> (dyn.Value, error)`
    
    And equivalent functions that take a previously constructed `dyn.Path`:
    * `dyn.GetByPath(value, dyn.Path) -> (dyn.Value, error)`
    * `dyn.SetByPath(value, dyn.Path, newValue) -> (dyn.Value, error)`
    * `dyn.MapByPath(value, dyn.Path, func) -> (dyn.Value, error)`
    
    Whenever the "set" and "map" functions need to change a value, they return
    a new `dyn.Value` such that the input value is never changed.
    pietern committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    d0d7536 View commit details
    Browse the repository at this point in the history
  3. Add dynvar package for variable resolution with a dyn.Value tree

    This is the `dyn` counterpart to the `bundle/config/interpolation` package.
    
    It relies on the paths in `${foo.bar}` being valid `dyn.Path` instances. It
    leverages `dyn.Walk` to get a complete picture of all variable references and
    uses `dyn.Get` to retrieve values pointed to by variable references.
    
    # Please enter the commit message for your changes. Lines starting
    # with '#' will be ignored, and an empty message aborts the commit.
    #
    # On branch dynvar
    # Changes to be committed:
    #	new file:   libs/dyn/dynvar/lookup.go
    #	new file:   libs/dyn/dynvar/lookup_test.go
    #	new file:   libs/dyn/dynvar/ref.go
    #	new file:   libs/dyn/dynvar/ref_test.go
    #	new file:   libs/dyn/dynvar/resolve.go
    #	new file:   libs/dyn/dynvar/resolve_test.go
    #
    # Untracked files:
    #	extra/
    #
    pietern committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    af0e3e2 View commit details
    Browse the repository at this point in the history
  4. Comments

    pietern committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    37bd8f1 View commit details
    Browse the repository at this point in the history
  5. Remove path and key from ref

    pietern committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    a7348d5 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    b739338 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    a20a7f9 View commit details
    Browse the repository at this point in the history
  8. Fixes

    pietern committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    36eb46e View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    764489e View commit details
    Browse the repository at this point in the history

Commits on Jan 25, 2024

  1. Configuration menu
    Copy the full SHA
    48e2737 View commit details
    Browse the repository at this point in the history
  2. Remove file

    pietern committed Jan 25, 2024
    Configuration menu
    Copy the full SHA
    b6ade6c View commit details
    Browse the repository at this point in the history

Commits on Jan 31, 2024

  1. Configuration menu
    Copy the full SHA
    0b79685 View commit details
    Browse the repository at this point in the history
  2. Fix test

    pietern committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    1b70822 View commit details
    Browse the repository at this point in the history
  3. Remove mergo dependency

    pietern committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    71f326f View commit details
    Browse the repository at this point in the history
  4. Make linter happy

    pietern committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    2dec864 View commit details
    Browse the repository at this point in the history

Commits on Feb 2, 2024

  1. Configuration menu
    Copy the full SHA
    1fbed61 View commit details
    Browse the repository at this point in the history

Commits on Feb 5, 2024

  1. Configuration menu
    Copy the full SHA
    01a8b20 View commit details
    Browse the repository at this point in the history
  2. Use dyn.Map for sync section

    pietern committed Feb 5, 2024
    Configuration menu
    Copy the full SHA
    fc9c31c View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    cdcf875 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    055a10e View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    afd7450 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    b6b3e07 View commit details
    Browse the repository at this point in the history
  7. Rename

    pietern committed Feb 5, 2024
    Configuration menu
    Copy the full SHA
    20d5380 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    c445d19 View commit details
    Browse the repository at this point in the history
  9. No new var

    pietern committed Feb 5, 2024
    Configuration menu
    Copy the full SHA
    6b78191 View commit details
    Browse the repository at this point in the history
  10. Newline

    pietern committed Feb 5, 2024
    Configuration menu
    Copy the full SHA
    9c88a7e View commit details
    Browse the repository at this point in the history
  11. Include PR link

    pietern committed Feb 5, 2024
    Configuration menu
    Copy the full SHA
    36666d8 View commit details
    Browse the repository at this point in the history
  12. Undo

    pietern committed Feb 5, 2024
    Configuration menu
    Copy the full SHA
    6b5fcb7 View commit details
    Browse the repository at this point in the history

Commits on Feb 6, 2024

  1. Ensure every variable reference is passed to lookup function

    References to keys that themselves are also variable references were
    shortcircuited in the previous approach. This meant that certain fields were
    resolved even if the lookup function would have instructed to skip resolution.
    
    To fix this we separate memoization of resolved variable references from
    memoization of lookups. Now, every variable reference is passed through the
    lookup function.
    pietern committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    eb0b6a2 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    49b9681 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    ac07608 View commit details
    Browse the repository at this point in the history
  4. .

    pietern committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    98f8344 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    44e7914 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    4f2c288 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    f7b3253 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    5f85d59 View commit details
    Browse the repository at this point in the history

Commits on Feb 7, 2024

  1. Configuration menu
    Copy the full SHA
    7bf5abf View commit details
    Browse the repository at this point in the history
  2. Zero destination struct in convert.ToTyped

    Not doing this means that the output struct is not a true representation
    of the `dyn.Value` and unrepresentable state (e.g. unexported fields)
    can be carried over across `convert.ToTyped` calls.
    pietern committed Feb 7, 2024
    Configuration menu
    Copy the full SHA
    9b60240 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    e330189 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    879959a View commit details
    Browse the repository at this point in the history
  5. Fix integration test

    pietern committed Feb 7, 2024
    Configuration menu
    Copy the full SHA
    d5c710f View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    f97fd5f View commit details
    Browse the repository at this point in the history

Commits on Feb 8, 2024

  1. Configuration menu
    Copy the full SHA
    3cd824a View commit details
    Browse the repository at this point in the history
  2. Omitempty for Git fields

    pietern committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    4406361 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    061d57b View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    61bc57e View commit details
    Browse the repository at this point in the history
  5. Clean up

    pietern committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    245b481 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    978f313 View commit details
    Browse the repository at this point in the history
  7. Eliminate panics

    pietern committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    7154f36 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    1c1fc57 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    36646e8 View commit details
    Browse the repository at this point in the history

Commits on Feb 13, 2024

  1. Retain partially valid structs in convert.Normalize

    Before this change, any error in a subtree would cause the entire subtree to be
    dropped from the output.
    
    This is not ideal when debugging, so instead we drop only the values that
    cannot be normalized. Note that this doesn't change behavior if the caller is
    properly checking the returned diagnostics for errors.
    pietern committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    d1f558f View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    160c371 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    727beca View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    938423f View commit details
    Browse the repository at this point in the history
  5. Rename

    pietern committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    8738ad6 View commit details
    Browse the repository at this point in the history
  6. Rename

    pietern committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    1a45a8c View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    be21195 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    0c47f4d View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    f8e701d View commit details
    Browse the repository at this point in the history

Commits on Feb 14, 2024

  1. Rename

    pietern committed Feb 14, 2024
    Configuration menu
    Copy the full SHA
    b72c75d View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    60bd1ec View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    0d59f4f View commit details
    Browse the repository at this point in the history

Commits on Feb 15, 2024

  1. Add option to include fields present in the type but not in the value

    This feature supports variable lookups in a `dyn.Value` that are present in the
    type but haven't been initialized with a value.
    
    For example: `${bundle.git.origin_url}` is present in the `dyn.Value` only if
    it was assigned a value. If it wasn't assigned a value it should resolve to the
    empty string. This normalization option, when set, ensures that all fields that
    are represented in the specified type are present in the return value.
    pietern committed Feb 15, 2024
    Configuration menu
    Copy the full SHA
    2fd8f7e View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    e6fc04f View commit details
    Browse the repository at this point in the history
  3. Skip instead of panic

    pietern committed Feb 15, 2024
    Configuration menu
    Copy the full SHA
    8774a96 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    c7ab94b View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    c2cd815 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    6495c10 View commit details
    Browse the repository at this point in the history
  7. Adapt updated origin/main

    pietern committed Feb 15, 2024
    Configuration menu
    Copy the full SHA
    3521d02 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    dcc4990 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    22f8a8e View commit details
    Browse the repository at this point in the history

Commits on Feb 16, 2024

  1. Configuration menu
    Copy the full SHA
    9d17138 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    184907c View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    2fa4e11 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    b740ec7 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    4e5605f View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    89961be View commit details
    Browse the repository at this point in the history