diff --git a/README.md b/README.md index ccf9b52..479411f 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,220 @@ # KubeSecondaryDNS DNS for KubeVirt VirtualMachines secondary interfaces +## Prerequisites +1. The KubeSecondaryDNS Deployment should be reachable from outside the cluster. +2. The guest must have a guest agent installed, so the IPs of the secondary interfaces will be reported. +3. Kubevirt must be installed, else the plugin would exit with error after 2 minutes, due to +the informer logic. It will be recreated according k8s logic. +4. If necessary, establish connectivity to cluster public IP via a relevant DNS entity that is used to +reach the authoritative KubeSecondaryDNS server, such as DNSResolver, TLD NameServer, etc. + +When running KubeSecondaryDNS on top on kubevirtci, we achieve [1] as follows: +1. Create a NodePort for the Deployment, use `31111` as the port. +2. Kubevirtci forwards all traffic from host `127.0.0.1:31111` to the kubevirtci container. +3. Traffic in the container, on port `31111` is forwarded to the control plane node of the cluster. + +This way `nslookup -port=31111 127.0.0.1` from outside the cluster can reach the secondary-dns container. + +## Deployment +Choose the desired release from +https://github.com/kubevirt/kubesecondarydns/releases +and apply its manifest. +For example: +```shell +KSD_RELEASE=v0.0.5 +URL=https://github.com/kubevirt/kubesecondarydns/releases/download/${KSD_RELEASE}/secondarydns.yaml +kubectl apply -f ${URL} +``` + +To undeploy it, use: +```shell +kubectl delete -f ${URL} +``` + +## Parameters +You can update those parameters in the manifest. + +`DOMAIN` (default: `""`) - Determines the suffix of the FQDN. +Can be used in order to differentiate between different clusters (by including the cluster name as part of it for example). +The FQDN would look as follows: +If `DOMAIN` is not empty `...vm.` +If `DOMAIN` is empty `...vm` + +`NAME_SERVER_IP` (default: `""`) - TODO + +## Development + +### Main operations +```shell +# run unit tests +make test + +# bring up a local cluster with Kubernetes +make cluster-up + +# build and deploy KubeSecondaryDNS +make cluster-sync + +# run e2e tests on the cluster, requires cluster with installed operator, +make functest + +# clean up all resources created by the operator from the cluster +make cluster-clean + +# delete the cluster +make cluster-down +``` + +### Cluster access +```shell +# access kubernetes API on the cluster +./cluster/kubectl.sh get nodes + +# ssh into the cluster's node +./cluster/cli.sh ssh node01 +``` + +### Additional operations (done by cluster-sync) +```shell +# build image +make build + +# push the image to the registry +make push + +# deploy KubeSecondaryDNS manifest +make deploy + +# create the node port +make create-nodeport +``` + +### Code related operations +```shell +# perform auto-formatting on the source code (if not done by your IDE) +make fmt + +# run go vet +make vet +``` + ### Mapping a container image to the code In order to know which git commit hash was used to build a certain container image perform the following steps: -```bash +```shell podman inspect --format '{{ index .Config.Labels "multi.GIT_SHA"}}' ``` Note that it can also be inspected by skopeo without downloading the image. + +## Basic example + +Deploy the following manifests and wait until the guest agent reports the IP, +and that the plugin refreshes its database (1-2 minutes). + +Resolve the FQDN using this command: +`nslookup -port=31111 nic1.vmi-sec.secondary-test.vm 127.0.0.1` +Note: In this example `DOMAIN` is an empty string. + +Expected output: +``` +Server: 127.0.0.1 +Address: 127.0.0.1#31111 + +Name: nic1.vmi-sec.secondary-test.vm +Address: 10.10.0.5 +``` + +Manifests: +``` +--- +apiVersion: v1 +kind: Namespace +metadata: + name: secondary-test +--- +apiVersion: k8s.cni.cncf.io/v1 +kind: NetworkAttachmentDefinition +metadata: + name: ptp-conf + namespace: secondary-test +spec: + config: '{ + "cniVersion": "0.3.1", + "name": "mynet", + "plugins": [ + { + "type": "ptp", + "ipam": { + "type": "static", + "addresses": [ + { + "address": "10.10.0.5/24", + "gateway": "10.10.0.254" + } + ] + } + } + ] + }' +--- +apiVersion: kubevirt.io/v1 +kind: VirtualMachineInstance +metadata: + name: vmi-sec + namespace: secondary-test +spec: + domain: + devices: + disks: + - disk: + bus: virtio + name: containerdisk + - disk: + bus: virtio + name: cloudinitdisk + interfaces: + - bridge: {} + name: nic1 + features: + acpi: + enabled: true + machine: + type: q35 + resources: + requests: + memory: 128Mi + networks: + - multus: + networkName: ptp-conf + name: nic1 + volumes: + - containerDisk: + image: kubevirt/cirros-container-disk-demo:latest + imagePullPolicy: Always + name: containerdisk + - cloudInitNoCloud: + userData: | + #!/bin/sh + echo 'printed from cloud-init userdata' + name: cloudinitdisk +``` + +## High level design + +![design](images/design.png) + +The KubeSecondaryDNS Deployment consists of two containers: +1. CoreDNS (https://coredns.io) +2. StatusMonitor container (includes ZoneManager module). + +StatusMonitor is watching VMI status changes via k8s API, +and sending the info to the ZoneManager about the secondary interfaces. +The ZoneManager builds the Zone File that the CoreDNS expects according to the received status. + +The Zone File consists of: +1. SOA header (DNS parameters) +2. List of FQDN and associated IP addresses. + +ZoneManager updates the `/zones` folder which is shared between the containers, with the updated data. +CoreDNS monitors the `/zones` folder, and updates its db accordingly (using CoreDNS `auto` plugin). +This way the CoreDNS can serve secondary interfaces DNS queries. diff --git a/images/design.png b/images/design.png new file mode 100644 index 0000000..a7ddec4 Binary files /dev/null and b/images/design.png differ