Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add kind cluster create script #791

Merged
merged 33 commits into from
Sep 1, 2019
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
bd053e4
add kind cluster
xiaojingchen Aug 20, 2019
110a2cd
add default clustername
xiaojingchen Aug 21, 2019
10393af
fix script
xiaojingchen Aug 21, 2019
025b5a4
address comment
xiaojingchen Aug 21, 2019
5ff3b8b
change doc
xiaojingchen Aug 22, 2019
4709a44
fix
xiaojingchen Aug 22, 2019
9bfeafa
address comment
xiaojingchen Aug 26, 2019
b6b51cb
add usage and command line options
xiaojingchen Aug 26, 2019
c7f1040
fix
xiaojingchen Aug 26, 2019
3dc0ada
Merge branch 'master' into kind-cluster
cofyc Aug 27, 2019
c4dbc9e
add git version limit
xiaojingchen Aug 27, 2019
9c9ce9f
Merge branch 'kind-cluster' of https://github.com/xiaojingchen/tidb-o…
xiaojingchen Aug 27, 2019
429d7af
address comment
xiaojingchen Aug 27, 2019
62d847d
fix typo
xiaojingchen Aug 27, 2019
6e9372a
address comment
xiaojingchen Aug 27, 2019
61246e1
typo
xiaojingchen Aug 27, 2019
af9fa6d
Merge branch 'master' into kind-cluster
cofyc Aug 28, 2019
6707176
Merge branch 'master' into kind-cluster
xiaojingchen Aug 28, 2019
69cbefa
fix filename
xiaojingchen Aug 28, 2019
2019445
Merge branch 'kind-cluster' of https://github.com/xiaojingchen/tidb-o…
xiaojingchen Aug 28, 2019
6230bdd
add help command
xiaojingchen Aug 28, 2019
8f0e9b3
mkdir workdir
xiaojingchen Aug 30, 2019
d65175a
Merge branch 'master' into kind-cluster
xiaojingchen Aug 30, 2019
265d621
refine doc
xiaojingchen Aug 30, 2019
ce91393
Merge branch 'kind-cluster' of https://github.com/xiaojingchen/tidb-o…
xiaojingchen Aug 30, 2019
cb8f949
remove useless info
xiaojingchen Aug 30, 2019
d21f9aa
Update docs/CONTRIBUTING.md
xiaojingchen Sep 1, 2019
8f8b49b
Update docs/CONTRIBUTING.md
xiaojingchen Sep 1, 2019
63505b5
Update docs/CONTRIBUTING.md
xiaojingchen Sep 1, 2019
52c3383
Update docs/CONTRIBUTING.md
xiaojingchen Sep 1, 2019
b0a3197
Update docs/CONTRIBUTING.md
xiaojingchen Sep 1, 2019
817ef41
refine script
xiaojingchen Sep 1, 2019
c5b9944
Merge branch 'master' into kind-cluster
xiaojingchen Sep 1, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 33 additions & 1 deletion docs/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,9 @@ $ make test

#### Run e2e tests

For e2e tests, we recommend DinD Kubernetes environment. Follow [this guide](./local-dind-tutorial.md) to spin up a local DinD Kubernetes cluster.
##### Run e2e tests in dind

First we should build a DinD Kubernetes environment. Follow [this guide](./local-dind-tutorial.md) to spin up a local DinD Kubernetes cluster.

Then you can build and push Docker images to the DinD Docker registry. The DinD Docker registry is available as `localhost:5000` both on the host machine and inside the DinD.

Expand All @@ -133,6 +135,36 @@ $ kubectl -n=tidb-operator-e2e logs -f tidb-operator-e2e

To re-run e2e tests, delete the testing pod and apply it again.

##### Run e2e tests in kind

Execute the follow command to create a [kind](https://kind.sigs.k8s.io/) Kubernetes environment.
xiaojingchen marked this conversation as resolved.
Show resolved Hide resolved

```sh
$ ./hack/kind-cluster-build.sh
```

Then you can build and push Docker images to the public Docker registry (There have not docker registry in Kind Kubernetes environment currently)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docker images can be loaded with kind load docker-image or kind load image-archive command.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can push images to all nodes, but if ImagePullPolicy is Always, the pod still can not get image from local. it can not resolve the image problem completely

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think kind load also load image to all nodes. For image always pull issue, can we use IfNotPresent to replace Always.

xiaojingchen marked this conversation as resolved.
Show resolved Hide resolved

```sh
$ export DOCKER_REGISTRY=<pulic-docker-registry>
$ make docker-push
$ make e2e-docker-push
```

After Docker images are pushed to the public Docker registry, run e2e tests:
xiaojingchen marked this conversation as resolved.
Show resolved Hide resolved

```sh
$ sed -i "s#127.0.0.1:5000/pingcap/tidb-operator-e2e:latest#${DOCKER_REGISTRY}/pingcap/tidb-operator-e2e:latest#" tests/manifests/e2e/e2e.yaml
$ sed -i "s#pingcap/tidb-operator:latest#${DOCKER_REGISTRY}/pingcap/tidb-operator:latest#" tests/manifests/e2e/e2e.yaml
$ kubectl apply -f tests/manifests/e2e/e2e.yaml
```

You can get the e2e test report from the log of testing pod:

```sh
$ kubectl -n=tidb-operator-e2e logs -f tidb-operator-e2e
```

### Step 5: Keep your branch in sync

While on your `myfeature` branch, run the following commands:
Expand Down
133 changes: 133 additions & 0 deletions hack/kind-cluster-build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
#!/usr/bin/env bash
set -e

usage() {
cat <<EOF
This script use kind to create Kubernetes cluster,about kind please refer: https://kind.sigs.k8s.io/
Before run this script,please ensure that:
* have installed git (version >= v2.15.1)
xiaojingchen marked this conversation as resolved.
Show resolved Hide resolved
* have installed docker

Options:
-h,--help prints the usage message
-n,--name name of the Kubernetes cluster,default value: kind
-c,--nodeNum the count of the cluster nodes,default value: 6
-k,--k8sVersion version of the Kubernetes cluster,default value: v1.12.8
-v,--volumeNum the volumes number of each kubernetes node,default value: 9
Usage:
$0 --name testCluster --nodeNum 4 --k8sVersion v1.12.9
EOF
}

while [[ $# -gt 0 ]]
do
key="$1"

case $key in
-n|--name)
clusterName="$2"
shift
shift
;;
-c|--nodeNum)
nodeNum="$2"
shift
shift
;;
-k|--k8sVersion)
k8sVersion="$2"
shift
shift
;;
-v|--volumeNum)
volumeNum="$2"
shift
shift
;;
-h|--help)
usage
exit 0
;;
*)
echo "unknown option: $key"
usage
exit 1
;;
esac
done

clusterName=${clusterName:-kind}
nodeNum=${nodeNum:-6}
k8sVersion=${k8sVersion:-v1.12.8}
volumeNum=${volumeNum:-9}

echo "clusterName: ${clusterName}"
echo "nodeNum: ${nodeNum}"
echo "k8sVersion: ${k8sVersion}"
echo "volumeNum: ${volumeNum}"

# check requirements
for requirement in kind kubectl helm docker
do
echo "############ check ${requirement} ##############"
if hash ${requirement} 2>/dev/null;then
echo "${requirement} have installed"
else
echo "this script needs ${requirement}, please install ${requirement} first."
exit 1
fi
done

echo "############# start create cluster:[${clusterName}] #############"

data_dir=${HOME}/kind/${clusterName}/data

echo "clean data dir: ${data_dir}"
if [ -d ${data_dir} ]; then
rm -rf ${data_dir}
fi

configFile=${HOME}/kind/${clusterName}/kind-config.yaml

cat <<EOF > ${configFile}
kind: Cluster
apiVersion: kind.sigs.k8s.io/v1alpha3
networking:
disableDefaultCNI: true
nodes:
- role: control-plane
EOF

for ((i=0;i<${nodeNum};i++))
do
mkdir -p ${data_dir}/worker${i}
tennix marked this conversation as resolved.
Show resolved Hide resolved
cat <<EOF >> ${configFile}
- role: worker
extraMounts:
EOF
for ((k=1;k<=${volumeNum};k++))
do
mkdir -p ${data_dir}/worker${i}/vol${k}
cat <<EOF >> ${configFile}
- containerPath: /mnt/disks/vol${k}
hostPath: ${data_dir}/worker${i}/vol${k}
EOF
done
done

echo "start to create k8s cluster"
kind create cluster --config ${configFile} --image kindest/node:${k8sVersion} --name=${clusterName}
export KUBECONFIG="$(kind get kubeconfig-path --name=${clusterName})"

echo "init tidb-operator env"
kubectl apply -f manifests/local-dind/kube-flannel.yaml
kubectl apply -f manifests/local-dind/local-volume-provisioner.yaml
kubectl apply -f manifests/tiller-rbac.yaml
kubectl apply -f manifests/crd.yaml
kubectl create ns tidb-operator-e2e
helm init --service-account=tiller --wait

echo "############# success create cluster:[${clusterName}] #############"

echo "To start using your cluster, run:"
echo " export KUBECONFIG=$(kind get kubeconfig-path --name=${clusterName})"