forked from dagster-io/dagster
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[time-window-partitions][perf] stop validating when unnecessary (dags…
…ter-io#15550) ## Summary & Motivation When we create a TimeWindowPartitionsSubset, there's an implicit contract that it doesn't contain invalid partition keys. Sometimes, we create this subset using time windows directly, but sometimes (especially in the partition mapping case), we create this subset with explicit keys (which then need to be converted into time windows later on). This short-circuits some of the checking logic in the conversion from regular partition string to time window. This is especially important in cases where the start_offset / end_offset are non-zero, as these end up generating very expensive calls to get_last_partition_window / get_first_partition_window ## How I Tested These Changes
- Loading branch information
1 parent
8c75d6e
commit 8c1ec5c
Showing
2 changed files
with
19 additions
and
27 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters