[Release-1.25] - K3s should not perform etcd setup when managed etcd is not enabled

[dereknola]

Backport fix for K3s should not perform etcd setup when managed etcd is not enabled

• K3s should not perform etcd setup when managed etcd is not enabled #3404

[mdrahman-suse]

Validated on release-1.25 branch with commit f7ab577

Environment Details

Infrastructure

• Cloud
• Hosted

Node(s) CPU architecture, OS, and Version:

PRETTY_NAME="Ubuntu 22.04.2 LTS"

Cluster Configuration:

2 servers, 1 agent
Split role setup: 1 etcd-only, 1 cp-only and 1 agent node

Config.yaml:

# etcd-only S1
disable-apiserver: true
disable-controller-manager: true
disable-scheduler: true
node-taint:
  - node-role.kubernetes.io/etcd:NoExecute
write-kubeconfig-mode: 644
cluster-init: true

# cp-only S2
disable-etcd: true
node-taint:
  - node-role.kubernetes.io/control-plane:NoSchedule
server: "https://<IP of S1>:6443"
token: "<K10 from S1>"
write-kubeconfig-mode: 644

# agent
server: "https://<IP of S1>:6443"
token: "<K10 from S1>"

Testing Steps

1. Copy config.yaml

$ sudo mkdir -p /etc/rancher/k3s && sudo cp config.yaml /etc/rancher/k3s

2. Install k3s on server and agent nodes
3. Ensure the cluster is up and running
4. Validate the files below are NOT present on the cp-only (as it has the flag disable-etcd: true) server node

$ sudo ls -la /var/lib/rancher/k3s/server/tls/etcd
...
-rw-r--r-- 1 root root 1258 Jul 18 16:16 server-client.crt
-rw------- 1 root root  227 Jul 18 16:16 server-client.key

Replication Results:

• k3s version used for replication:

k3s version v1.25.11+k3s1 (582f07cf)
go version go1.19.10

• Observed the files are present on cp-only node

$ sudo ls -la /var/lib/rancher/k3s/server/tls/etcd
total 48
drwx------ 2 root root 4096 Jul 18 16:16 .
drwx------ 4 root root 4096 Jul 18 16:16 ..
-rw-r--r-- 1 root root 1140 Jul 18 16:16 client.crt
-rw------- 1 root root  227 Jul 18 16:16 client.key
-rw------- 1 root root  566 Jul 18 16:00 peer-ca.crt
-rw------- 1 root root  227 Jul 18 16:00 peer-ca.key
-rw-r--r-- 1 root root 1250 Jul 18 16:16 peer-server-client.crt
-rw------- 1 root root  227 Jul 18 16:16 peer-server-client.key
-rw------- 1 root root  570 Jul 18 16:00 server-ca.crt
-rw------- 1 root root  227 Jul 18 16:00 server-ca.key
-rw-r--r-- 1 root root 1258 Jul 18 16:16 server-client.crt
-rw------- 1 root root  227 Jul 18 16:16 server-client.key

Validation Results:

• k3s version used for validation:

k3s version v1.25.11+k3s-f7ab577c (f7ab577c)
go version go1.19.10

• Observed the files are NOT present on cp-only node

$ sudo ls -la /var/lib/rancher/k3s/server/tls/etcd
total 40
drwx------ 2 root root 4096 Jul 18 17:57 .
drwx------ 4 root root 4096 Jul 18 17:57 ..
-rw-r--r-- 1 root root 1144 Jul 18 17:57 client.crt
-rw------- 1 root root  227 Jul 18 17:57 client.key
-rw------- 1 root root  566 Jul 18 17:51 peer-ca.crt
-rw------- 1 root root  227 Jul 18 17:51 peer-ca.key
-rw-r--r-- 1 root root 1250 Jul 18 17:57 peer-server-client.crt
-rw------- 1 root root  227 Jul 18 17:57 peer-server-client.key
-rw------- 1 root root  574 Jul 18 17:51 server-ca.crt
-rw------- 1 root root  227 Jul 18 17:51 server-ca.key