Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update credentials when using ADC in Compute Engine #23773

Merged
merged 3 commits into from
Jun 3, 2022
Merged

Update credentials when using ADC in Compute Engine #23773

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
25009f6
Update credentials when using ADC in Compute Engine
MaksYermak May 17, 2022
56d7778
Add try-catch for 404 error
MaksYermak May 19, 2022
be06fe7
Move 'refresh' method to '_get_credentials_email' in base_google hook
MaksYermak Jun 3, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion airflow/providers/google/common/hooks/base_google.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,10 @@
import tenacity
from google.api_core.exceptions import Forbidden, ResourceExhausted, TooManyRequests
from google.api_core.gapic_v1.client_info import ClientInfo
from google.auth import _cloud_sdk
from google.auth import _cloud_sdk, compute_engine
from google.auth.environment_vars import CLOUD_SDK_CONFIG_DIR, CREDENTIALS
from google.auth.exceptions import RefreshError
from google.auth.transport import _http_client
from googleapiclient import discovery
from googleapiclient.errors import HttpError
from googleapiclient.http import MediaIoBaseDownload, build_http, set_user_agent
Expand Down Expand Up @@ -274,6 +276,17 @@ def _get_credentials_email(self) -> str:
If user authentication (e.g. gcloud auth) is used, it returns the e-mail account of that user.
"""
credentials = self._get_credentials()

if isinstance(credentials, compute_engine.Credentials):
try:
credentials.refresh(_http_client.Request())
except RefreshError as msg:
"""
If the Compute Engine metadata service can't be reached in this case the instance has not
credentials.
"""
self.log.debug(msg)

service_account_email = getattr(credentials, 'service_account_email', None)
if service_account_email:
return service_account_email
Expand Down