Skip to content

Commit

Permalink
Update Kubernetes library version (#18797)
Browse files Browse the repository at this point in the history
Previously we pinned this version as v12 as a change to Kube library
internals meant v1.Pod objects now have a logger object inside them, and
couldn't be pickled on Python 3.6.

To fix that we have "backported" the change in Python 3.7 to make Logger
objects be pickled "by name". (In Python 3.7 the change adds
`__reduce__` methods on to the Logger and RootLogger objects, but here
we achieve it `copyreg` stdlib module so we don't monkeypatch
anything.)

This fix is also applied in to airflow core in a separate commit, but we
also apply it here in the provider so that cncf.kubernetes client
library can be updated but still used with older versions of Airflow
that don't have this fix in.

(cherry picked from commit 7222f68)
  • Loading branch information
ashb authored and ephraimbuddy committed Mar 26, 2022
1 parent fb34b22 commit 30243e9
Show file tree
Hide file tree
Showing 2 changed files with 11 additions and 3 deletions.
8 changes: 6 additions & 2 deletions airflow/providers/cncf/kubernetes/utils/pod_manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,11 @@
from airflow.utils.log.logging_mixin import LoggingMixin

if TYPE_CHECKING:
from kubernetes.client.models.core_v1_event_list import CoreV1EventList
try:
# Kube >= 19
from kubernetes.client.models.core_v1_event_list import CoreV1EventList as V1EventList
except ImportError:
from kubernetes.client.models.v1_event_list import V1EventList


class PodLaunchFailedException(AirflowException):
Expand Down Expand Up @@ -317,7 +321,7 @@ def read_pod_logs(
raise

@tenacity.retry(stop=tenacity.stop_after_attempt(3), wait=tenacity.wait_exponential(), reraise=True)
def read_pod_events(self, pod: V1Pod) -> "CoreV1EventList":
def read_pod_events(self, pod: V1Pod) -> "V1EventList":
"""Reads events from the POD"""
try:
return self._client.list_namespaced_event(
Expand Down
6 changes: 5 additions & 1 deletion tests/kubernetes/test_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -59,5 +59,9 @@ def test_disable_verify_ssl(self):

_disable_verify_ssl()

configuration = Configuration()
# Support wide range of kube client libraries
if hasattr(Configuration, 'get_default_copy'):
configuration = Configuration.get_default_copy()
else:
configuration = Configuration()
self.assertFalse(configuration.verify_ssl)

0 comments on commit 30243e9

Please sign in to comment.