IPv4/IPv6 dual stack docs (kubernetes#16010)

* initial commit for IPv4/IPv6 dual stack docs

Signed-off-by: Lachlan Evenson <lachlan.evenson@microsoft.com>

* Apply suggestions from code review

Co-Authored-By: Tim Bannister <tim@scalefactory.com>

* Remove warning, Add What's next section

Signed-off-by: Lachlan Evenson <lachlan.evenson@microsoft.com>

* Add Service section
Add Provising a dual stack Kubernetes cluster section
Add Ecosystem tooling section
Update prerequisites
Update flags
Update supported features
Move validation to task
Add Service validation

Signed-off-by: Lachlan Evenson <lachlan.evenson@microsoft.com>

* Apply suggestions from code review

Co-Authored-By: Tim Bannister <tim@scalefactory.com>

* Remove ecosystem tooling
Remove provisioning tools
Add backtics to ipFamily values
Update loadbalancer section

Signed-off-by: Lachlan Evenson <lachlan.evenson@microsoft.com>

* Fix feature gate link typo

Co-Authored-By: Tim Bannister <tim@scalefactory.com>

* Update to dual-stack
Add default use-case to Service validation
Add note to default Service behaviour
Add default Service example
Update egress routing description

Signed-off-by: Lachlan Evenson <lachlan.evenson@microsoft.com>

* Update api-server to the API server
Fix small typo based on feedback

Co-Authored-By: Tim Bannister <tim@scalefactory.com>
Signed-off-by: Lachlan Evenson <lachlan.evenson@microsoft.com>

* Add for_k8s_version feature state

Signed-off-by: Lachlan Evenson <lachlan.evenson@microsoft.com>

* Update service IP address verbiage to be more concise

Co-Authored-By: Tim Bannister <tim@scalefactory.com>

* Move to tasks/network

Signed-off-by: Lachlan Evenson <lachlan.evenson@microsoft.com>

* Move dual-stack under services-networking

Signed-off-by: Lachlan Evenson <lachlan.evenson@microsoft.com>

* Remove dual-stack from glossary
Add codenew blocks
Split command from output
Renamed pod name
Created subheading to validate node and pod addressing

Signed-off-by: Lachlan Evenson <lachlan.evenson@microsoft.com>

* Apply suggestions from code review

Co-Authored-By: Tim Bannister <tim@scalefactory.com>

* Verbiage update based on review

Co-Authored-By: Tim Bannister <tim@scalefactory.com>
Signed-off-by: Lachlan Evenson <lachlan.evenson@microsoft.com>

* Apply suggestions from code review

Co-Authored-By: Tim Bannister <tim@scalefactory.com>

content/en/docs/concepts/services-networking/dual-stack.md

@@ -0,0 +1,111 @@
+---
+reviewers:
+- lachie83
+- khenidak
+- aramase
+title: IPv4/IPv6 dual-stack
+feature:
+  title: IPv4/IPv6 dual-stack
+  description: >
+    Allocation of IPv4 and IPv6 addresses to Pods and Services
+
+content_template: templates/concept
+weight: 70
+---
+
+{{% capture overview %}}
+
+{{< feature-state for_k8s_version="v1.16" state="alpha" >}}
+
+ IPv4/IPv6 dual-stack enables the allocation of both IPv4 and IPv6 addresses to {{< glossary_tooltip text="Pods" term_id="pod" >}} and {{< glossary_tooltip text="Services" term_id="service" >}}.
+
+If you enable IPv4/IPv6 dual-stack networking for your Kubernetes cluster, the cluster will support the simultaneous assignment of both IPv4 and IPv6 addresses.
+
+{{% /capture %}}
+
+{{% capture body %}}
+
+## Supported Features
+
+Enabling IPv4/IPv6 dual-stack on your Kubernetes cluster provides the following features:
+
+   * Dual-stack Pod networking (a single IPv4 and IPv6 address assignment per Pod)
+   * IPv4 and IPv6 enabled Services (each Service must be for a single address family)
+   * Kubenet multi address family support (IPv4 and IPv6)
+   * Pod off-cluster egress routing (eg. the Internet) via both IPv4 and IPv6 interfaces
+
+## Prerequisites
+
+The following prerequisites are needed in order to utilize IPv4/IPv6 dual-stack Kubernetes clusters:
+
+   * Kubernetes 1.16 or later
+   * Provider support for dual-stack networking (Cloud provider or otherwise must be able to provide Kubernetes nodes with routable IPv4/IPv6 network interfaces)
+   * Kubenet network plugin
+   * Kube-proxy running in mode IPVS
+
+## Enable IPv4/IPv6 dual-stack
+
+To enable IPv4/IPv6 dual-stack, enable the `IPv6DualStack` [feature gate](https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/) for the relevant components of your cluster, and set dual-stack cluster network assignments:
+
+   * kube-controller-manager:
+      * `--feature-gates="IPv6DualStack=true"`
+      * `--cluster-cidr=<IPv4 CIDR>,<IPv6 CIDR>` eg. `--cluster-cidr=10.244.0.0/16,fc00::/24`
+      * `--service-cluster-ip-range=<IPv4 CIDR>,<IPv6 CIDR>`
+   * kubelet:
+      * `--feature-gates="IPv6DualStack=true"`
+   * kube-proxy:
+      * `--proxy-mode=ipvs`
+      * `--cluster-cidrs=<IPv4 CIDR>,<IPv6 CIDR>` 
+      * `--feature-gates="IPv6DualStack=true"`
+
+{{< caution >}}
+If you specify an IPv6 address block larger than a /24 via  `--cluster-cidr` on the command line, that assignment will fail.
+{{< /caution >}}
+
+## Services
+
+If your cluster has IPv4/IPv6 dual-stack networking enabled, you can create {{< glossary_tooltip text="Services" term_id="service" >}} with either an IPv4 or an IPv6 address. You can choose the address family for the Service's cluster IP by setting a field, `.spec.ipFamily`, on that Service.
+You can only set this field when creating a new Service. Setting the `.spec.ipFamily` field is optional and should only be used if you plan to enable IPv4 and IPv6 {{< glossary_tooltip text="Services" term_id="service" >}} and {{< glossary_tooltip text="Ingresses" term_id="ingress" >}} on your cluster. The configuration of this field not a requirement for [egress](#egress-traffic) traffic.
+
+{{< note >}}
+The default address family for your cluster is the address family of the first service cluster IP range configured via the `--service-cluster-ip-range` flag to the kube-controller-manager.
+{{< /note >}}
+
+You can set `.spec.ipFamily` to either:
+
+   * `IPv4`: The API server will assign an IP from a `service-cluster-ip-range` that is `ipv4`
+   * `IPv6`: The API server will assign an IP from a `service-cluster-ip-range` that is `ipv6`
+
+The following Service specification does not include the `ipFamily` field. Kubernetes will assign an IP address (also known as a "cluster IP") from the first configured `service-cluster-ip-range` to this Service.
+
+{{< codenew file="service/networking/dual-stack-default-svc.yaml" >}}
+
+The following Service specification includes the `ipFamily` field. Kubernetes will assign an IPv6 address (also known as a "cluster IP") from the configured `service-cluster-ip-range` to this Service.
+
+{{< codenew file="service/networking/dual-stack-ipv6-svc.yaml" >}}
+
+For comparison, the following Service specification will be assigned an IPV4 address (also known as a "cluster IP") from the configured `service-cluster-ip-range` to this Service.
+
+{{< codenew file="service/networking/dual-stack-ipv4-svc.yaml" >}}
+
+### Type LoadBalancer
+
+On cloud providers which support IPv6 enabled external load balancers, setting the `type` field to `LoadBalancer` in additional to setting `ipFamily` field to `IPv6` provisions a cloud load balancer for your Service.
+
+## Egress Traffic
+
+The use of publicly routable and non-publicly routable IPv6 address blocks is acceptable provided the underlying {{< glossary_tooltip text="CNI" term_id="cni" >}} provider is able to implement the transport. If you have a Pod that uses non-publicly routable IPv6 and want that Pod to reach off-cluster destinations (eg. the public Internet), you must set up IP masquerading for the egress traffic and any replies. The [ip-masq-agent](https://github.com/kubernetes-incubator/ip-masq-agent) is dual-stack aware, so you can use ip-masq-agent for IP masquerading on dual-stack clusters.
+
+## Known Issues
+
+   * IPv6 network block assignment uses the default IPv4 CIDR block size (/24)
+   * Kubenet forces IPv4,IPv6 positional reporting of IPs (--cluster-cidr)
+   * Dual-stack networking does not function if the `EndpointSlice` feature gate is enabled.
+
+{{% /capture %}}
+
+{{% capture whatsnext %}}
+
+* [Validate IPv4/IPv6 dual-stack](/docs/tasks/network/validate-dual-stack) networking
+
+{{% /capture %}}

content/en/docs/tasks/network/_index.md

@@ -0,0 +1,5 @@
+---
+title: "Network"
+weight: 160
+---
+

content/en/docs/tasks/network/validate-dual-stack.md

@@ -0,0 +1,126 @@
+---
+reviewers:
+- lachie83
+- khenidak
+title: Validate IPv4/IPv6 dual-stack
+content_template: templates/task
+---
+
+{{% capture overview %}}
+This document shares how to validate IPv4/IPv6 dual-stack enabled Kubernetes clusters.
+{{% /capture %}}
+
+{{% capture prerequisites %}}
+
+* Kubernetes 1.16 or later
+* Provider support for dual-stack networking (Cloud provider or otherwise must be able to provide Kubernetes nodes with routable IPv4/IPv6 network interfaces)
+* Kubenet network plugin
+* Kube-proxy running in mode IPVS
+* [Dual-stack enabled](/docs/concepts/services-networking/dual-stack/) cluster
+
+{{% /capture %}}
+
+{{% capture steps %}}
+
+## Validate addressing
+
+### Validate node addressing
+
+Each dual-stack Node should have a single IPv4 block and a single IPv6 block allocated. Validate that IPv4/IPv6 Pod address ranges are configured by running the following command. Replace the sample node name with a valid dual-stack Node from your cluster. In this example, the Node's name is `k8s-linuxpool1-34450317-0`:
+
+```shell
+kubectl get nodes k8s-linuxpool1-34450317-0 -o go-template --template='{{range .spec.podCIDRs}}{{printf "%s\n" .}}{{end}}'
+```
+```
+10.244.1.0/24
+a00:100::/24
+```
+There should be one IPv4 block and one IPv6 block allocated.
+
+Validate that the node has an IPv4 and IPv6 interface detected (replace node name with a valid node from the cluster. In this example the node name is k8s-linuxpool1-34450317-0): 
+```shell
+kubectl get nodes k8s-linuxpool1-34450317-0 -o go-template --template='{{range .status.addresses}}{{printf "%s: %s \n" .type .address}}{{end}}'
+```
+```
+Hostname: k8s-linuxpool1-34450317-0
+InternalIP: 10.240.0.5
+InternalIP: 2001:1234:5678:9abc::5
+```
+
+### Validate Pod addressing
+
+Validate that a Pod has an IPv4 and IPv6 address assigned. (replace the Pod name with a valid Pod in your cluster. In this example the Pod name is pod01)
+```shell
+kubectl get pods pod01 -o go-template --template='{{range .status.podIPs}}{{printf "%s \n" .ip}}{{end}}'
+```
+```
+10.244.1.4
+a00:100::4
+```
+
+## Validate Services
+
+Create the following Service without the `ipFamily` field set. When this field is not set, the Service gets an IP from the first configured range via `--service-cluster-ip-range` flag on the kube-controller-manager.
+
+{{< codenew file="service/networking/dual-stack-default-svc.yaml" >}}
+
+By viewing the YAML for the Service you can observe that the Service has the `ipFamily` field has set to reflect the address family of the first configured range set via `--service-cluster-ip-range` flag on kube-controller-manager.
+
+```shell
+kubectl get svc my-service -o yaml
+```
+
+```yaml
+apiVersion: v1
+kind: Service
+metadata:
+  creationTimestamp: "2019-09-03T20:45:13Z"
+  labels:
+    app: MyApp
+  name: my-service
+  namespace: default
+  resourceVersion: "485836"
+  selfLink: /api/v1/namespaces/default/services/my-service
+  uid: b6fa83ef-fe7e-47a3-96a1-ac212fa5b030
+spec:
+  clusterIP: 10.0.29.179
+  ipFamily: IPv4
+  ports:
+  - port: 80
+    protocol: TCP
+    targetPort: 9376
+  selector:
+    app: MyApp
+  sessionAffinity: None
+  type: ClusterIP
+status:
+  loadBalancer: {}
+```
+
+Create the following Service with the `ipFamily` field set to `IPv6`.
+
+{{< codenew file="service/networking/dual-stack-ipv6-svc.yaml" >}}
+
+Validate that the Service gets a cluster IP address from the IPv6 address block. You may then validate access to the service via the IP and port.
+```
+ kubectl get svc -l app=MyApp
+NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
+my-service   ClusterIP   fe80:20d::d06b   <none>        80/TCP    9s
+```
+
+### Create a dual-stack load balanced Service
+
+If the cloud provider supports the provisioning of IPv6 enabled external load balancer, create the following Service with both the `ipFamily` field set to `IPv6` and the `type` field set to `LoadBalancer`
+
+{{< codenew file="service/networking/dual-stack-ipv6-lb-svc.yaml" >}}
+
+Validate that the Service receives a `CLUSTER-IP` address from the IPv6 address block along with an `EXTERNAL-IP`. You may then validate access to the service via the IP and port. 
+```
+ kubectl get svc -l app=MyApp
+NAME         TYPE        CLUSTER-IP       EXTERNAL-IP                     PORT(S)        AGE
+my-service   ClusterIP   fe80:20d::d06b   2001:db8:f100:4002::9d37:c0d7   80:31868/TCP   30s
+```
+
+{{% /capture %}}
+
+

content/en/examples/service/networking/dual-stack-default-svc.yaml

@@ -0,0 +1,11 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: my-service
+spec:
+  selector:
+    app: MyApp
+  ports:
+    - protocol: TCP
+      port: 80
+      targetPort: 9376

content/en/examples/service/networking/dual-stack-ipv4-svc.yaml

@@ -0,0 +1,12 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: my-service
+spec:
+  ipFamily: IPv4
+  selector:
+    app: MyApp
+  ports:
+    - protocol: TCP
+      port: 80
+      targetPort: 9376

content/en/examples/service/networking/dual-stack-ipv6-lb-svc.yaml

@@ -0,0 +1,15 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: my-service
+  labels:
+    app: MyApp
+spec:
+  ipFamily: IPv6
+  type: LoadBalancer
+  selector:
+    app: MyApp
+  ports:
+    - protocol: TCP
+      port: 80
+      targetPort: 9376

content/en/examples/service/networking/dual-stack-ipv6-svc.yaml

@@ -0,0 +1,12 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: my-service
+spec:
+  ipFamily: IPv6
+  selector:
+    app: MyApp
+  ports:
+    - protocol: TCP
+      port: 80
+      targetPort: 9376

data/concepts.yml

@@ -95,6 +95,7 @@ toc:
   - docs/concepts/services-networking/ingress.md
   - docs/concepts/services-networking/network-policies.md
   - docs/concepts/services-networking/add-entries-to-pod-etc-hosts-with-host-aliases.md
+  - docs/concepts/services-networking/dual-stack.md
 
 - title: Storage
   landing_page: /docs/concepts/storage/volumes/

data/tasks.yml

@@ -126,6 +126,11 @@ toc:
   - docs/tasks/tls/managing-tls-in-a-cluster.md
   - docs/tasks/tls/certificate-rotation.md
 
+- title: Network
+  landing_page: tasks/network/validate-dual-stack/
+  section:
+  - docs/tasks/network/validate-dual-stack.md
+
 - title: Administer a Cluster
   landing_page: /docs/tasks/administer-cluster/memory-default-namespace/
   section: