Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make fileset take optional list of paths to list #1684

Merged
merged 6 commits into from
Aug 19, 2024
Merged

Conversation

pietern
Copy link
Contributor

@pietern pietern commented Aug 15, 2024

Changes

Before this change, the fileset library would take a single root path and list all files in it. To support an allowlist of paths to list (much like a Git pathspec without patterns; see pathspec), this change introduces an optional argument to fileset.New where the caller can specify paths to list. If not specified, this argument defaults to list . (i.e. list all files in the root).

The motivation for this change is that we wish to expose this pattern in bundles. Users should be able to specify which paths to synchronize instead of always only synchronizing the bundle root directory.

Tests

New and existing unit tests.

func New(root vfs.Path) *FileSet {
// It optionally accepts a list of paths relative to the root to include in the fileset.
// If not specified, it defaults to including all files in the root path.
func New(root vfs.Path, args ...[]string) *FileSet {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why slice of slices of strings instead of just slices of strings?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nvm, I see that the usage with such syntax is a bit more readable

@pietern pietern added this pull request to the merge queue Aug 19, 2024
Merged via the queue into main with commit 7de7583 Aug 19, 2024
5 checks passed
@pietern pietern deleted the fileset-paths branch August 19, 2024 15:22
github-merge-queue bot pushed a commit that referenced this pull request Aug 19, 2024
## Changes

Requires #1684. 

## Tests

Ran the sync integration tests.
andrewnester added a commit that referenced this pull request Aug 21, 2024
CLI:
 * Added filtering flags for cluster list commands ([#1703](#1703)).

Bundles:
 * Remove reference to "dbt" in the default-sql template ([#1696](#1696)).
 * Pause continuous pipelines when 'mode: development' is used ([#1590](#1590)).
 * Add configurable presets for name prefixes, tags, etc. ([#1490](#1490)).
 * Report all empty resources present in error diagnostic ([#1685](#1685)).
 * Improves detection of PyPI package names in environment dependencies ([#1699](#1699)).
 * [DAB] Add support for requirements libraries in Job Tasks ([#1543](#1543)).
 * Add paths field to bundle sync configuration ([#1694](#1694)).

Internal:
 * Add `import` option for PyDABs ([#1693](#1693)).
 * Make fileset take optional list of paths to list ([#1684](#1684)).
 * Pass through paths argument to libs/sync ([#1689](#1689)).
 * Correctly mark package names with versions as remote libraries ([#1697](#1697)).
 * Share test initializer in common helper function ([#1695](#1695)).
 * Make `pydabs/venv_path` optional ([#1687](#1687)).
 * Use API mocks for duplicate path errors in workspace files extensions client ([#1690](#1690)).
 * Fix prefix preset used for UC schemas ([#1704](#1704)).
github-merge-queue bot pushed a commit that referenced this pull request Aug 22, 2024
CLI:
* Added filtering flags for cluster list commands
([#1703](#1703)).

Bundles:
* Remove reference to "dbt" in the default-sql template
([#1696](#1696)).
* Pause continuous pipelines when 'mode: development' is used
([#1590](#1590)).
* Add configurable presets for name prefixes, tags, etc.
([#1490](#1490)).
* Report all empty resources present in error diagnostic
([#1685](#1685)).
* Improves detection of PyPI package names in environment dependencies
([#1699](#1699)).
* [DAB] Add support for requirements libraries in Job Tasks
([#1543](#1543)).
* Add paths field to bundle sync configuration
([#1694](#1694)).

Internal:
* Add `import` option for PyDABs
([#1693](#1693)).
* Make fileset take optional list of paths to list
([#1684](#1684)).
* Pass through paths argument to libs/sync
([#1689](#1689)).
* Correctly mark package names with versions as remote libraries
([#1697](#1697)).
* Share test initializer in common helper function
([#1695](#1695)).
* Make `pydabs/venv_path` optional
([#1687](#1687)).
* Use API mocks for duplicate path errors in workspace files extensions
client ([#1690](#1690)).
* Fix prefix preset used for UC schemas
([#1704](#1704)).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants