-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Alternative dev monday #1
Conversation
|
||
@flow(**all_flow_kwargs) | ||
def run_dbt_flow(): #Why not inside of the flow? | ||
dbt_config=DbtConfig( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this should be passed by the user of the lib to the dbt_flow
func
prefect_dbt_flow_2/utils/tasks.py
Outdated
from typing import List, Dict, Optional | ||
|
||
from prefect import task | ||
from prefect_dbt.cli.commands import trigger_dbt_cli_command |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i'd rather not depend on prefect_db
t for now
prefect_dbt_flow_2/utils/tasks.py
Outdated
name=dbt_node.name, | ||
# retries="", #Do we need this? it goes in dbt_config of prefect config? | ||
# retry_delay_seconds="", | ||
).submint( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what does submint
do?
Why are you passing the _task_dbt_run
to another task? Don't see how this would work?
Summary:
I have made changes to the way we configure dbt tasks and the project structure.
Project Structure:
prefect_dbt_flow_2
flow.py
(entry point)utils
__init__.py
(contains classes)flow.py
(creates the Prefect flow)tasks.py
(handles node execution)graph.py
(parses nodes from dbt)cmd.py
(runs commands)