Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow control plane static pods #120

Merged
merged 4 commits into from
May 26, 2021
Merged

Allow control plane static pods #120

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
1bbba04
Allow control plane static pods
bjw-s May 26, 2021
da7d8c6
Rename vars, path
bjw-s May 26, 2021
d834ca1
Merge branch 'main' into staticpods
bjw-s May 26, 2021
32c68ea
Update README.md
bjw-s May 26, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 17 additions & 16 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,22 +61,23 @@ below.
Below are variables that are set against all of the play hosts for environment
consistency. These are generally cluster-level configuration.

| Variable | Description | Default Value |
|----------------------------------|------------------------------------------------------------------------------------|--------------------------------|
| `k3s_state` | State of k3s: installed, started, stopped, downloaded, uninstalled, validated. | installed |
| `k3s_release_version` | Use a specific version of k3s, eg. `v0.2.0`. Specify `false` for stable. | `false` |
| `k3s_config_file` | Location of the k3s configuration file. | `/etc/rancher/k3s/config.yaml` |
| `k3s_build_cluster` | When multiple play hosts are available, attempt to cluster. Read notes below. | `true` |
| `k3s_registration_address` | Fixed registration address for nodes. IP or FQDN. | NULL |
| `k3s_github_url` | Set the GitHub URL to install k3s from. | https://github.com/k3s-io/k3s |
| `k3s_install_dir` | Installation directory for k3s. | `/usr/local/bin` |
| `k3s_install_hard_links` | Install using hard links rather than symbolic links. | `false` |
| `k3s_server_manifests_templates` | A list of Auto-Deploying Manifests Templates (only deploys on primary controller). | [] |
| `k3s_use_experimental` | Allow the use of experimental features in k3s. | `false` |
| `k3s_use_unsupported_config` | Allow the use of unsupported configurations in k3s. | `false` |
| `k3s_etcd_datastore` | Enable etcd embedded datastore (read notes below). | `false` |
| `k3s_debug` | Enable debug logging on the k3s service. | `false` |
| `k3s_registries` | Registries configuration file content. | `{ mirrors: {}, configs:{} }` |
| Variable | Description | Default Value |
|---------------------------------------|------------------------------------------------------------------------------------|--------------------------------|
| `k3s_state` | State of k3s: installed, started, stopped, downloaded, uninstalled, validated. | installed |
| `k3s_release_version` | Use a specific version of k3s, eg. `v0.2.0`. Specify `false` for stable. | `false` |
| `k3s_config_file` | Location of the k3s configuration file. | `/etc/rancher/k3s/config.yaml` |
| `k3s_build_cluster` | When multiple play hosts are available, attempt to cluster. Read notes below. | `true` |
| `k3s_registration_address` | Fixed registration address for nodes. IP or FQDN. | NULL |
| `k3s_github_url` | Set the GitHub URL to install k3s from. | https://github.com/k3s-io/k3s |
| `k3s_install_dir` | Installation directory for k3s. | `/usr/local/bin` |
| `k3s_install_hard_links` | Install using hard links rather than symbolic links. | `false` |
| `k3s_server_manifests_templates` | A list of Auto-Deploying Manifests Templates (only deploys on primary controller). | [] |
| `k3s_server_staticpods_templates` | A list of templates for installing static pod manifests on the control plane. | [] |
bjw-s marked this conversation as resolved.
Show resolved Hide resolved
| `k3s_use_experimental` | Allow the use of experimental features in k3s. | `false` |
| `k3s_use_unsupported_config` | Allow the use of unsupported configurations in k3s. | `false` |
| `k3s_etcd_datastore` | Enable etcd embedded datastore (read notes below). | `false` |
| `k3s_debug` | Enable debug logging on the k3s service. | `false` |
| `k3s_registries` | Registries configuration file content. | `{ mirrors: {}, configs:{} }` |

### K3S Service Configuration

Expand Down
3 changes: 3 additions & 0 deletions defaults/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,9 @@ k3s_install_hard_links: false
# A list of templates used for preconfigure the cluster.
k3s_server_manifests_templates: []

# A list of templates used for installing static pod manifests on the control plane.
k3s_server_pod_manifests_templates: []

# Use experimental features in k3s?
k3s_use_experimental: false

Expand Down
18 changes: 18 additions & 0 deletions tasks/build/preconfigure-k3s-static-pod-manifests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
---

- name: Ensure that the pod-manifests directory exists
ansible.builtin.file:
state: directory
path: "{{ k3s_server_pod_manifests_dir }}"
mode: 0755
when: k3s_server_pod_manifests_templates | length > 0
become: "{{ k3s_become_for_directory_creation | ternary(true, false, k3s_become_for_all) }}"

# https://github.com/k3s-io/k3s/pull/1691
- name: Ensure static pod manifests are copied to controllers
ansible.builtin.template:
src: "{{ item }}"
dest: "{{ k3s_server_pod_manifests_dir }}/{{ item | basename | replace('.j2','') }}"
mode: 0644
loop: "{{ k3s_server_pod_manifests_templates }}"
become: "{{ k3s_become_for_directory_creation | ternary(true, false, k3s_become_for_all) }}"
5 changes: 5 additions & 0 deletions tasks/state-installed.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,11 @@
- k3s_primary_control_node
- k3s_server_manifests_templates | length > 0

- import_tasks: build/preconfigure-k3s-static-pod-manifests.yml
when:
- k3s_control_node
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense to make this k3s_primary_control_node ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It depends I guess? If you want to deploy Pods there (such as for kube-vip), I think the manifest should go on all of the control plane nodes.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are probably right, let's keep it this way. I know for the other manifest option it makes sense to put them on a single controller node. Static pods might behave differently.

Copy link

@toboshii toboshii May 26, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Static pods are directly tied to their kubelet (they're essentially a container outside of the cluster, represented by a mirror object inside the cluster) therefore they must be deployed individually to each control node.

- k3s_server_pod_manifests_templates | length > 0

- import_tasks: build/install-k3s.yml

- name: Ensure containerd installation tasks are run
Expand Down
4 changes: 4 additions & 0 deletions vars/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,10 @@ k3s_token_location: "{{ k3s_config_dir }}/cluster-token"
# https://rancher.com/docs/k3s/latest/en/advanced/#auto-deploying-manifests
k3s_server_manifests_dir: "{{ k3s_data_dir }}/server/manifests"

# Path for static pod manifests that are deployed on the control plane
# https://github.com/k3s-io/k3s/pull/1691
k3s_server_pod_manifests_dir: "{{ k3s_data_dir }}/agent/pod-manifests"

# Packages that we need to check are installed
k3s_check_packages: []
# - name: dummy
Expand Down