This folder contains Kubeflow Pipelines Kustomize manifests for a light weight deployment. You can follow the instruction and deploy Kubeflow Pipelines in an existing cluster.
To install Kubeflow Pipelines, you have several options.
- Via GCP AI Platform UI.
- Via an upcoming commandline tool.
- Via Kubectl with Kustomize, it's detailed here.
Deploy latest version of Kubeflow Pipelines.
It uses following default settings.
- image: latest released images
- namespace: kubeflow
- application name: pipeline
It's based on in-cluster PersistentVolumeClaim storage.
kubectl apply -k cluster-scoped-resources/
kubectl wait crd/applications.app.k8s.io --for condition=established --timeout=60s
kubectl apply -k env/platform-agnostic/
kubectl wait pods -l application-crd-id=kubeflow-pipelines -n kubeflow --for condition=Ready --timeout=1800s
kubectl port-forward -n kubeflow svc/ml-pipeline-ui 8080:80
Now you can access it via localhost:8080
It's based on in-cluster PersistentVolumeClaim storage. Additionally, it introduced a proxy in GCP to allow user easily access KFP safely.
kubectl apply -k cluster-scoped-resources/
kubectl wait crd/applications.app.k8s.io --for condition=established --timeout=60s
kubectl apply -k env/dev/
kubectl wait applications/pipeline -n kubeflow --for condition=Ready --timeout=1800s
# Or visit http://console.cloud.google.com/ai-platform/pipelines
kubectl describe configmap inverse-proxy-config -n kubeflow | grep googleusercontent.com
Its storage is based on CloudSQL & GCS. It's better than others for production usage.
Please following sample for a customized installation.
Its storage is based on S3 & AWS RDS. It's more natural for AWS users to use this option.
Please following AWS Instructions for installation.
Note: Community maintains a repo e2fyi/kubeflow-aws for AWS.
If the installation is based on CloudSQL/GCS, after the uninstall, the data is still there, reinstall a newer version can reuse the data.
### 1. namespace scoped
# Depends on how you installed it:
kubectl kustomize env/platform-agnostic | kubectl delete -f -
# or
kubectl kustomize env/dev | kubectl delete -f -
# or
kubectl kustomize env/gcp | kubectl delete -f -
# or
kubectl delete applications/pipeline -n kubeflow
### 2. cluster scoped
kubectl delete -k cluster-scoped-resources/
Run
kubectl create clusterrolebinding your-binding --clusterrole=cluster-admin --user=[your-user-name]
If sample code requires a "user-gcp-sa" secret, you could create one by
-
First download the GCE VM service account token Document
gcloud iam service-accounts keys create application_default_credentials.json \ --iam-account [SA-NAME]@[PROJECT-ID].iam.gserviceaccount.com
-
Run
kubectl create secret -n [your-namespace] generic user-gcp-sa --from-file=user-gcp-sa.json=application_default_credentials.json`
- User facing manifest entrypoints are
cluster-scoped-resources
package andenv/<env-name>
package.cluster-scoped-resources
should collect all cluster-scoped resources.env/<env-name>
should collect env specific namespace-scoped resources.- Note, for multi-user envs, they already included cluster-scoped resources.
- KFP core components live in
base/<component-name>
folders.- If a component requires cluster-scoped resources, it should have a folder inside named
cluster-scoped
with related resources, but note thatbase/<component-name>/kustomization.yaml
shouldn't include thecluster-scoped
folder.cluster-scoped
folders should be collected by top levelcluster-scoped-resources
folder.
- If a component requires cluster-scoped resources, it should have a folder inside named
- KFP core installations are in
base/installs/<install-type>
, they only include the core KFP components, not third party ones. - Third party components live in
third-party/<component-name>
folders.
Env specific overlays live in env/<env-name>
folders, they compose above components to get ready for directly deploying.
Please compose base/installs/<install-type>
and third party dependencies based on your own requirements.
Constraints for namespaced installation we need to comply with (that drove above structure):
- CRDs must be applied separately, because if we apply CRs in the same
kubectl apply
command, the CRD may not have been accepted by k8s api server (e.g. Application CRD). - A Kubeflow 1.0 constraint is that we should separate cluster scoped resources from namespace scoped resources, because sometimes different roles are required to deploy them. Cluster scoped resources usually need a cluster admin role, while namespaced resources can be deployed by individual teams managing a namespace.