Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not suppress normalisation diagnostics for resolving variables #1740

Merged
merged 2 commits into from
Sep 2, 2024

Conversation

andrewnester
Copy link
Contributor

Changes

Tested on the following bundle configuration

bundle:
  name: clusters
  mode: development

variables:
  webhook_notifications:
    description: Webhook URL for notifications
    type: complex
    default:
      on_failure:
        id: 6a6c04c1-389c-4534-95af-b68b62a9dbe6

resources:
  jobs:
    test_job:
      name: "Andrew Nester Test Job"
      tasks:
        - task_key: test_task
          notebook_task:
            notebook_path: "./src/test.py"
          new_cluster:
            num_workers: 2
            node_type_id: "i3.xlarge"
            autoscale:
              min_workers: 2
              max_workers: 7
            spark_version: "12.2.x-scala2.12"
            spark_conf:
              "spark.executor.memory": "2g"
      webhook_notifications: ${var.webhook_notifications}

bundle validate output is below

andrew.nester@HFW9Y94129 wheel % databricks bundle validate
Warning: expected sequence, found map
  at resources.jobs.test_job.webhook_notifications.on_failure
  in bundle.yml:11:9

Name: clusters
Target: default
Workspace:
  User: andrew.nester@databricks.com
  Path: /Users/andrew.nester@databricks.com/.bundle/clusters/default

Note that error correctly points to the variable

Copy link
Contributor

@pietern pietern left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

bundle/config/mutator/resolve_variable_references.go Outdated Show resolved Hide resolved
@andrewnester andrewnester added this pull request to the merge queue Sep 2, 2024
Merged via the queue into main with commit 582558c Sep 2, 2024
5 checks passed
@andrewnester andrewnester deleted the fix/var-normalise-diags branch September 2, 2024 09:26
andrewnester added a commit that referenced this pull request Sep 5, 2024
CLI:
 * Do not error if we cannot prompt for a profile in `auth login` ([#1745](#1745)).

Bundles:
 * Pass along $AZURE_CONFIG_FILE to Terraform process ([#1734](#1734)).
 * Add prompt when a pipeline recreation happens ([#1672](#1672)).
 * Use materialized views in the default-sql template ([#1709](#1709)).
 * Update templates to latest LTS DBR ([#1715](#1715)).
 * Make lock optional in the JSON schema ([#1738](#1738)).
 * Do not suppress normalisation diagnostics for resolving variables ([#1740](#1740)).
 * Include a permissions section in all templates ([#1713](#1713)).
 * Fixed complex variables are not being correctly merged from include files ([#1746](#1746)).
 * Fixed variable override in target with full variable syntax ([#1749](#1749)).

Internal:
 * Consider serverless clusters as compatible for Python wheel tasks ([#1733](#1733)).
 * PythonMutator: explain missing package error ([#1736](#1736)).
 * Add `dyn.Time` to box a timestamp with its original string value ([#1732](#1732)).
 * Fix streaming of stdout, stdin, stderr in cobra test runner ([#1742](#1742)).

Dependency updates:
 * Bump github.com/Masterminds/semver/v3 from 3.2.1 to 3.3.0 ([#1741](#1741)).
github-merge-queue bot pushed a commit that referenced this pull request Sep 5, 2024
CLI:
* Do not error if we cannot prompt for a profile in `auth login`
([#1745](#1745)).

Bundles:

As of this release CLI will show a prompt is if there are configuration
changes which will lead to a DLT recreation.
Users can skip the prompt by specifying the `--auto-approve` flag

* Pass along $AZURE_CONFIG_FILE to Terraform process
([#1734](#1734)).
* Add prompt when a pipeline recreation happens
([#1672](#1672)).
* Use materialized views in the default-sql template
([#1709](#1709)).
* Update templates to latest LTS DBR
([#1715](#1715)).
* Make lock optional in the JSON schema
([#1738](#1738)).
* Do not suppress normalisation diagnostics for resolving variables
([#1740](#1740)).
* Include a permissions section in all templates
([#1713](#1713)).
* Fixed complex variables are not being correctly merged from include
files ([#1746](#1746)).
* Fixed variable override in target with full variable syntax
([#1749](#1749)).

Internal:
* Consider serverless clusters as compatible for Python wheel tasks
([#1733](#1733)).
* PythonMutator: explain missing package error
([#1736](#1736)).
* Add `dyn.Time` to box a timestamp with its original string value
([#1732](#1732)).
* Fix streaming of stdout, stdin, stderr in cobra test runner
([#1742](#1742)).

Dependency updates:
* Bump github.com/Masterminds/semver/v3 from 3.2.1 to 3.3.0
([#1741](#1741)).

---------

Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants