Snapshot

Snapshot build of the main branch. Updated automatically on each commit.

v0.230.0

Notable changes for Databricks Asset Bundles:

Workspace paths are automatically prefixed with /Workspace. In addition, all usage of path strings such as /Workspace/${workspace.root_path}/... in bundle configuration is automatically replaced with ${workspace.root_path}/... and generates a warning as part of bundle validate.

More details can be found here: https://docs.databricks.com/en/release-notes/dev-tools/bundles.html#workspace-paths

Bundles:

• Add an error if state files grow bigger than the export limit (#1795).
• Always prepend bundle remote paths with /Workspace (#1724).
• Add resource path field to bundle workspace configuration (#1800).
• Add validation for files with a .(resource-name).yml extension (#1780).

Internal:

• Remove deprecated or readonly fields from the bundle schema (#1809).

API Changes:

• Changed databricks git-credentials create, databricks git-credentials delete, databricks git-credentials get, databricks git-credentials list, databricks git-credentials update commands .
• Changed databricks repos create, databricks repos delete, databricks repos get, databricks repos update command .

OpenAPI commit 0c86ea6dbd9a730c24ff0d4e509603e476955ac5 (2024-10-02)
Dependency updates:

• Upgrade TF provider to 1.53.0 (#1815).
• Bump golang.org/x/term from 0.24.0 to 0.25.0 (#1811).
• Bump golang.org/x/text from 0.18.0 to 0.19.0 (#1812).
• Bump github.com/databricks/databricks-sdk-go from 0.47.0 to 0.48.0 (#1810).

v0.229.0

Bundles:

• Added support for creating all-purpose clusters (#1698).
• Reduce time until the prompt is shown for bundle run (#1727).
• Use Unity Catalog for pipelines in the default-python template (#1766).
• Add verbose flag to the "bundle deploy" command (#1774).
• Fixed full variable override detection (#1787).
• Add sub-extension to resource files in built-in templates (#1777).
• Fix panic in apply_presets.go (#1796).

Internal:

• Assert tokens are redacted in origin URL when username is not specified (#1785).
• Refactor jobs path translation (#1782).
• Add JobTaskClusterSpec validate mutator (#1784).
• Pin Go toolchain to 1.22.7 (#1790).
• Modify SetLocation test utility to take full locations as argument (#1788).
• Simplified isFullVariableOverrideDef implementation (#1791).
• Sort tasks by task_key before generating the Terraform configuration (#1776).
• Trim trailing whitespace (#1794).
• Move trampoline code into trampoline package (#1793).
• Rename RootPath -> BundleRootPath (#1792).

API Changes:

• Changed databricks apps delete command to return .
• Changed databricks apps deploy command with new required argument order.
• Changed databricks apps start command to return .
• Changed databricks apps stop command to return .
• Added databricks temporary-table-credentials command group.
• Added databricks serving-endpoints put-ai-gateway command.
• Added databricks disable-legacy-access command group.
• Added databricks account disable-legacy-features command group.

OpenAPI commit 6f6b1371e640f2dfeba72d365ac566368656f6b6 (2024-09-19)
Dependency updates:

• Upgrade to Go SDK 0.47.0 (#1799).
• Upgrade to TF provider 1.52 (#1781).
• Bump golang.org/x/mod from 0.20.0 to 0.21.0 (#1758).
• Bump github.com/hashicorp/hc-install from 0.7.0 to 0.9.0 (#1772).

v0.228.1

Bundles:

• Added listing cluster filtering for cluster lookups (#1754).
• Expand library globs relative to the sync root (#1756).
• Fixed generated YAML missing 'default' for empty values (#1765).
• Use periodic triggers in all templates (#1739).
• Use the friendly name of service principals when shortening their name (#1770).
• Fixed detecting full syntax variable override which includes type field (#1775).

Internal:

• Pass copy of dyn.Path to callback function (#1747).
• Make bundle JSON schema modular with $defs (#1700).
• Alias variables block in the Target struct (#1748).
• Add end to end integration tests for bundle JSON schema (#1726).
• Fix artifact upload integration tests (#1767).

API Changes:

• Added databricks quality-monitors regenerate-dashboard command.

OpenAPI commit d05898328669a3f8ab0c2ecee37db2673d3ea3f7 (2024-09-04)
Dependency updates:

• Bump golang.org/x/term from 0.23.0 to 0.24.0 (#1757).
• Bump golang.org/x/oauth2 from 0.22.0 to 0.23.0 (#1761).
• Bump golang.org/x/text from 0.17.0 to 0.18.0 (#1759).
• Bump github.com/databricks/databricks-sdk-go from 0.45.0 to 0.46.0 (#1760).

v0.228.0

CLI:

• Do not error if we cannot prompt for a profile in auth login (#1745).

Bundles:

As of this release, the CLI will show a prompt if there are configuration changes that lead to DLT pipeline recreation.
Users can skip the prompt by specifying the --auto-approve flag.

• Pass along $AZURE_CONFIG_FILE to Terraform process (#1734).
• Add prompt when a pipeline recreation happens (#1672).
• Use materialized views in the default-sql template (#1709).
• Update templates to latest LTS DBR (#1715).
• Make lock optional in the JSON schema (#1738).
• Do not suppress normalisation diagnostics for resolving variables (#1740).
• Include a permissions section in all templates (#1713).
• Fixed complex variables are not being correctly merged from include files (#1746).
• Fixed variable override in target with full variable syntax (#1749).

Internal:

• Consider serverless clusters as compatible for Python wheel tasks (#1733).
• PythonMutator: explain missing package error (#1736).
• Add dyn.Time to box a timestamp with its original string value (#1732).
• Fix streaming of stdout, stdin, stderr in cobra test runner (#1742).

Dependency updates:

• Bump github.com/Masterminds/semver/v3 from 3.2.1 to 3.3.0 (#1741).

v0.227.1

CLI:

• Disable prompt for storage-credentials get command (#1723).

Bundles:

• Do not treat empty path as a local path (#1717).
• Correctly mark PyPI package name specs with multiple specifiers as remote libraries (#1725).
• Improve error handling for /Volumes paths in mode: development (#1716).

Internal:

• Ignore CLI version check on development builds of the CLI (#1714).

API Changes:

• Added databricks resource-quotas command group.
• Added databricks policy-compliance-for-clusters command group.
• Added databricks policy-compliance-for-jobs command group.

OpenAPI commit 3eae49b444cac5a0118a3503e5b7ecef7f96527a (2024-08-21)
Dependency updates:

• Bump github.com/databricks/databricks-sdk-go from 0.44.0 to 0.45.0 (#1719).
• Revert hc-install version to 0.7.0 (#1711).

v0.227.0

CLI:

• Added filtering flags for cluster list commands (#1703).

Bundles:

• Allow users to configure paths (including outside of the bundle root) to synchronize to the workspace. (#1694).
• Add configurable presets for name prefixes, tags, etc. (#1490).
• Add support for requirements libraries in Job Tasks (#1543).
• Remove reference to "dbt" in the default-sql template (#1696).
• Pause continuous pipelines when 'mode: development' is used (#1590).
• Report all empty resources present in error diagnostic (#1685).
• Improves detection of PyPI package names in environment dependencies (#1699).

Internal:

• Add import option for PyDABs (#1693).
• Make fileset take optional list of paths to list (#1684).
• Pass through paths argument to libs/sync (#1689).
• Correctly mark package names with versions as remote libraries (#1697).
• Share test initializer in common helper function (#1695).
• Make pydabs/venv_path optional (#1687).
• Use API mocks for duplicate path errors in workspace files extensions client (#1690).
• Fix prefix preset used for UC schemas (#1704).

v0.226.0

CLI:

• Add command line autocomplete to the fs commands (#1622).
• Add trailing slash to directory to produce completions for (#1666).
• Fix ability to import the CLI repository as module (#1671).
• Fix host resolution order in auth login (#1370).
• Print text logs in import-dir and export-dir commands (#1682).

Bundles:

• Expand and upload local wheel libraries for all task types (#1649).
• Clarify file format required for the config-file flag in bundle init (#1651).
• Fixed incorrectly cleaning up python wheel dist folder (#1656).
• Merge job parameters based on their name (#1659).
• Fix glob expansion after running a generic build command (#1662).
• Upload local libraries even if they don't have artifact defined (#1664).

Internal:

• Fix python wheel task integration tests (#1648).
• Skip pushing Terraform state after destroy (#1667).
• Enable Spark JAR task test (#1658).
• Run Spark JAR task test on multiple DBR versions (#1665).
• Stop tracking file path locations in bundle resources (#1673).
• Update VS Code settings to match latest value from IDE plugin (#1677).
• Use service.NamedIdMap to make lookup generation deterministic (#1678).
• [Internal] Remove dependency to the openapi package of the Go SDK (#1676).
• Upgrade TF provider to 1.50.0 (#1681).
• Upgrade Go SDK to 0.44.0 (#1679).

API Changes:

• Changed databricks account budgets create command . New request type is .
• Changed databricks account budgets create command to return .
• Changed databricks account budgets delete command . New request type is .
• Changed databricks account budgets delete command to return .
• Changed databricks account budgets get command . New request type is .
• Changed databricks account budgets get command to return .
• Changed databricks account budgets list command to require request of .
• Changed databricks account budgets list command to return .
• Changed databricks account budgets update command . New request type is .
• Changed databricks account budgets update command to return .
• Added databricks account usage-dashboards command group.
• Changed databricks model-versions get command to return .
• Changed databricks cluster-policies create command with new required argument order.
• Changed databricks cluster-policies edit command with new required argument order.
• Added databricks clusters update command.
• Added databricks genie command group.
• Changed databricks permission-migration migrate-permissions command . New request type is .
• Changed databricks permission-migration migrate-permissions command to return .
• Changed databricks account workspace-assignment delete command to return .
• Changed databricks account workspace-assignment update command with new required argument order.
• Changed databricks account custom-app-integration create command with new required argument order.
• Changed databricks account custom-app-integration list command to require request of .
• Changed databricks account published-app-integration list command to require request of .
• Removed databricks apps command group.
• Added databricks notification-destinations command group.
• Changed databricks shares list command to require request of .
• Changed databricks alerts create command . New request type is .
• Changed databricks alerts delete command . New request type is .
• Changed databricks alerts delete command to return .
• Changed databricks alerts get command with new required argument order.
• Changed databricks alerts list command to require request of .
• Changed databricks alerts list command to return .
• Changed databricks alerts update command . New request type is .
• Changed databricks alerts update command to return .
• Changed databricks queries create command . New request type is .
• Changed databricks queries delete command . New request type is .
• Changed databricks queries delete command to return .
• Changed databricks queries get command with new required argument order.
• Changed databricks queries list command to return .
• Removed databricks queries restore command.
• Changed databricks queries update command . New request type is .
• Added databricks queries list-visualizations command.
• Changed databricks query-visualizations create command . New request type is .
• Changed databricks query-visualizations delete command . New request type is .
• Changed databricks query-visualizations delete command to return .
• Changed databricks query-visualizations update command . New request type is .
• Changed databricks statement-execution execute-statement command to return .
• Changed databricks statement-execution get-statement command to return .
• Added databricks alerts-legacy command group.
• Added databricks queries-legacy command group.
• Added databricks query-visualizations-legacy command group.

OpenAPI commit f98c07f9c71f579de65d2587bb0292f83d10e55d (2024-08-12)
Dependency updates:

• Bump github.com/hashicorp/hc-install from 0.7.0 to 0.8.0 (#1652).
• Bump golang.org/x/sync from 0.7.0 to 0.8.0 (#1655).
• Bump golang.org/x/mod from 0.19.0 to 0.20.0 (#1654).
• Bump golang.org/x/oauth2 from 0.21.0 to 0.22.0 (#1653).
• Bump golang.org/x/text from 0.16.0 to 0.17.0 (#1670).
• Bump golang.org/x/term from 0.22.0 to 0.23.0 (#1669).

v0.225.0

Bundles:

• Add resource for UC schemas to DABs (#1413).

Internal:

• Use dynamic walking to validate unique resource keys (#1614).
• Regenerate TF schema (#1635).
• Add upgrade and upgrade eager flags to pip install call (#1636).
• Added test for negation pattern in sync include exclude section (#1637).
• Use precomputed terraform plan for bundle deploy (#1640).

v0.224.1

Bundles:

• Add UUID function to bundle template functions (#1612).
• Upgrade TF provider to 1.49.0 (#1617).
• Upgrade TF provider to 1.49.1 (#1626).
• Support multiple locations for diagnostics (#1610).
• Split artifact cleanup into prepare step before build (#1618).
• Move to a single prompt during bundle destroy (#1583).

Internal:

• Add tests for the Workspace API readahead cache (#1605).
• Update Python dependencies before install when upgrading a labs project (#1624).