Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GoogleDriveToLocalOperator gives conn_id error despite not taking conn_id as a param #24494

Closed
1 of 2 tasks
aspain opened this issue Jun 16, 2022 · 5 comments
Closed
1 of 2 tasks
Assignees
Labels
good first issue kind:bug This is a clearly a bug provider:google Google (including GCP) related issues

Comments

@aspain
Copy link
Contributor

aspain commented Jun 16, 2022

Apache Airflow version

2.3.2 (latest released)

What happened

When using the GoogleDriveToLocalOperator I receive the below error:

  File "/usr/local/lib/python3.9/site-packages/airflow/models/connection.py", line 430, in get_connection_from_secrets
    raise AirflowNotFoundException(f"The conn_id `{conn_id}` isn't defined")
airflow.exceptions.AirflowNotFoundException: The conn_id `google_cloud_default` isn't defined

but it doesn't appear that you can add a gcp_conn_id parameter - I receive an error if I try to do this

What you think should happen instead

Not entirely sure but possibly this operator should have a gcp_conn_id param similar to the GoogleDriveToGCSOperator

How to reproduce

Set up a task similar to the below - ensuring the service account has access to the shared folder being accessed

    download_from_gdrive_to_local = GoogleDriveToLocalOperator(
        task_id="download_from_gdrive_to_local",
        folder_id="0ACWYrvwOWlMsUk9PVA",
        file_name="abc.txt",
        output_file="/tmp/abc.txt",
    )

Operating System

macOS 11.5.2

Versions of Apache Airflow Providers

apache-airflow-providers-google - version 6.8.0

Deployment

Astronomer

Deployment details

using astro-runtime:5.0.3

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@aspain aspain added area:core kind:bug This is a clearly a bug labels Jun 16, 2022
@josh-fell
Copy link
Contributor

josh-fell commented Jun 16, 2022

Indeed this operator is missing a gcp_conn_id param to pass to the GoogleDriveHook. By default the gcp_conn_id used is "google_cloud_default" in the hook, but it can't be overwritten. I assume in this situation there was a conn ID setup that isn't named "google_cloud_default".

Would you like to take a crack at this one @aspain?

@josh-fell josh-fell added provider:google Google (including GCP) related issues good first issue and removed area:core labels Jun 16, 2022
@aspain
Copy link
Contributor Author

aspain commented Jun 16, 2022

👍 will take a stab at it

@antasmishra
Copy link

I too faced almost a similar issue when using google_cloud_default. When pressing the 'Test' button, there is a red area on top with no message. On [Right Click] + [Inspect] on area, it says The conn_id test isn't defined

@aspain
Copy link
Contributor Author

aspain commented Jun 23, 2022

@josh-fell I created PR #24622 to resolve this

@aspain aspain closed this as completed Jun 23, 2022
@aspain aspain reopened this Jun 23, 2022
@aspain
Copy link
Contributor Author

aspain commented Jun 30, 2022

PR has been merged 👍

@aspain aspain closed this as completed Jun 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue kind:bug This is a clearly a bug provider:google Google (including GCP) related issues
Projects
None yet
Development

No branches or pull requests

3 participants