initial commit
This commit is contained in:
@@ -0,0 +1,245 @@
|
||||
# Kubernetes Cluster Deployment with Ansible
|
||||
This repository contains Ansible playbooks to set up a Kubernetes cluster on existing Ubuntu 24.04 VMs.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. Three Ubuntu 24.04 (Noble Numbat) VMs running
|
||||
2. SSH access to all VMs
|
||||
3. Ansible installed on your control machine
|
||||
4. Sudo access configured on all VMs
|
||||
|
||||
## Repository Structure
|
||||
|
||||
```
|
||||
kubernetes-ansible/
|
||||
├── inventory.yml # Inventory file with VM information
|
||||
├── main.yml # Main playbook with task orchestration
|
||||
├── tasks/
|
||||
│ ├── setup-hosts.yml # Configure /etc/hosts
|
||||
│ ├── update-dns.yml # Configure DNS
|
||||
│ ├── ssh.yml # Configure SSH settings
|
||||
│ ├── setup-kernel.yml # Configure kernel modules and sysctl
|
||||
│ ├── node-setup.yml # Install Kubernetes components
|
||||
│ ├── control-plane-init.yml # Initialize control plane
|
||||
│ └── worker-join.yml # Join worker nodes to cluster
|
||||
└── README.md
|
||||
```
|
||||
|
||||
## Configuration Steps
|
||||
|
||||
1. Clone this repository:
|
||||
```bash
|
||||
git clone <repository-url>
|
||||
cd kubernetes-ansible
|
||||
```
|
||||
|
||||
2. Install required Ansible collections:
|
||||
```bash
|
||||
ansible-galaxy collection install community.general
|
||||
```
|
||||
|
||||
3. Edit `inventory.yml` and update the IP addresses of your VMs:
|
||||
```yaml
|
||||
controlplane:
|
||||
ansible_host: 192.168.1.X # Replace with your control plane IP
|
||||
node01:
|
||||
ansible_host: 192.168.1.Y # Replace with your first worker node IP
|
||||
node02:
|
||||
ansible_host: 192.168.1.Z # Replace with your second worker node IP
|
||||
```
|
||||
|
||||
4. Update the SSH credentials in the inventory file:
|
||||
```yaml
|
||||
vars:
|
||||
ansible_user: ubuntu # Replace with your SSH user
|
||||
# Use one of these depending on your authentication method:
|
||||
# ansible_ssh_private_key_file: ~/.ssh/id_rsa
|
||||
# ansible_ssh_pass: your_password
|
||||
```
|
||||
|
||||
## Deployment
|
||||
|
||||
You can deploy the entire cluster at once:
|
||||
|
||||
```bash
|
||||
ansible-playbook -i inventory.yml main.yml
|
||||
```
|
||||
|
||||
Or run deployment incrementally using tags:
|
||||
|
||||
```bash
|
||||
# 1. Configure hosts, DNS, SSH, and kernel
|
||||
ansible-playbook -i inventory.yml main.yml --tags common
|
||||
|
||||
# 2. Install Kubernetes components on all nodes
|
||||
ansible-playbook -i inventory.yml main.yml --tags k8s-setup
|
||||
|
||||
# 3. Initialize control plane
|
||||
ansible-playbook -i inventory.yml main.yml --tags control-plane
|
||||
|
||||
# 4. Join worker nodes
|
||||
ansible-playbook -i inventory.yml main.yml --tags workers
|
||||
```
|
||||
|
||||
## Iterative Testing Approach
|
||||
|
||||
To methodically test the deployment step by step:
|
||||
|
||||
### 1. Test Basic Connectivity
|
||||
|
||||
```bash
|
||||
# Verify SSH access to all nodes
|
||||
ansible -i inventory.yml all -m ping
|
||||
```
|
||||
|
||||
### 2. Test Basic System Configuration
|
||||
|
||||
```bash
|
||||
# Apply common configuration to control plane only
|
||||
ansible-playbook -i inventory.yml main.yml --tags common --limit controlplane
|
||||
|
||||
# Verify hosts file configuration
|
||||
ansible -i inventory.yml controlplane -m command -a "cat /etc/hosts"
|
||||
|
||||
# Verify DNS configuration
|
||||
ansible -i inventory.yml controlplane -m command -a "systemctl status systemd-resolved"
|
||||
```
|
||||
|
||||
### 3. Test Kernel Configuration
|
||||
|
||||
```bash
|
||||
# Verify kernel modules
|
||||
ansible -i inventory.yml controlplane -m command -a "lsmod | grep -E 'overlay|br_netfilter'"
|
||||
|
||||
# Verify sysctl settings
|
||||
ansible -i inventory.yml controlplane -m command -a "sysctl net.bridge.bridge-nf-call-iptables net.ipv4.ip_forward"
|
||||
```
|
||||
|
||||
### 4. Test Kubernetes Components
|
||||
|
||||
```bash
|
||||
# Install Kubernetes components on control plane
|
||||
ansible-playbook -i inventory.yml main.yml --tags k8s-setup --limit controlplane
|
||||
|
||||
# Verify containerd is running
|
||||
ansible -i inventory.yml controlplane -m command -a "systemctl status containerd"
|
||||
|
||||
# Verify Kubernetes binaries
|
||||
ansible -i inventory.yml controlplane -m command -a "kubectl version --client"
|
||||
```
|
||||
|
||||
### 5. Initialize Control Plane
|
||||
|
||||
```bash
|
||||
# Initialize control plane
|
||||
ansible-playbook -i inventory.yml main.yml --tags control-plane
|
||||
|
||||
# Verify control plane initialization
|
||||
ansible -i inventory.yml controlplane -m command -a "kubectl --kubeconfig=/etc/kubernetes/admin.conf get nodes"
|
||||
```
|
||||
|
||||
### 6. Join Worker Nodes One by One
|
||||
|
||||
```bash
|
||||
# Join first worker node
|
||||
ansible-playbook -i inventory.yml main.yml --tags workers --limit node01
|
||||
|
||||
# Verify first worker joined
|
||||
ansible -i inventory.yml controlplane -m command -a "kubectl --kubeconfig=/etc/kubernetes/admin.conf get nodes"
|
||||
|
||||
# Join second worker node
|
||||
ansible-playbook -i inventory.yml main.yml --tags workers --limit node02
|
||||
|
||||
# Verify all nodes
|
||||
ansible -i inventory.yml controlplane -m command -a "kubectl --kubeconfig=/etc/kubernetes/admin.conf get nodes"
|
||||
```
|
||||
|
||||
### 7. Test Network Plugin
|
||||
|
||||
```bash
|
||||
# Check Flannel pods
|
||||
ansible -i inventory.yml controlplane -m command -a "kubectl --kubeconfig=/etc/kubernetes/admin.conf get pods -n kube-system | grep flannel"
|
||||
|
||||
# Deploy test application
|
||||
ansible -i inventory.yml controlplane -m command -a "kubectl --kubeconfig=/etc/kubernetes/admin.conf create deployment nginx --image=nginx"
|
||||
ansible -i inventory.yml controlplane -m command -a "kubectl --kubeconfig=/etc/kubernetes/admin.conf expose deployment nginx --port=80 --type=NodePort"
|
||||
```
|
||||
|
||||
### 8. Debugging Common Issues
|
||||
|
||||
If any step fails, try these debugging commands:
|
||||
|
||||
```bash
|
||||
# Check service logs
|
||||
ansible -i inventory.yml <node> -m command -a "journalctl -xeu <service-name>"
|
||||
|
||||
# Example for kubelet
|
||||
ansible -i inventory.yml node01 -m command -a "journalctl -xeu kubelet"
|
||||
|
||||
# Check network connectivity
|
||||
ansible -i inventory.yml controlplane -m command -a "ping -c 1 {{ hostvars['node01']['ansible_host'] }}"
|
||||
```
|
||||
|
||||
### 9. Reset if Needed
|
||||
|
||||
To reset a node and start fresh:
|
||||
|
||||
```bash
|
||||
# Reset kubeadm
|
||||
ansible -i inventory.yml <node> -m command -a "kubeadm reset -f"
|
||||
|
||||
# Clean up directories
|
||||
ansible -i inventory.yml <node> -m command -a "rm -rf /etc/kubernetes /var/lib/kubelet /var/lib/etcd ~/.kube"
|
||||
```
|
||||
|
||||
## Key Components
|
||||
|
||||
- **Kernel Configuration**: Sets up required kernel modules (overlay, br_netfilter) and sysctl settings
|
||||
- **Container Runtime**: Installs and configures containerd with systemd cgroup driver
|
||||
- **Kubernetes**: Installs kubelet, kubeadm, and kubectl
|
||||
- **Networking**: Configures Flannel as the CNI plugin with pod CIDR 10.244.0.0/16
|
||||
|
||||
## Ubuntu 24.04 Specific Notes
|
||||
|
||||
- This playbook has been adjusted for Ubuntu 24.04 (Noble Numbat)
|
||||
- The containerd installation includes a fallback to the Docker repository if the default package isn't available
|
||||
- Kernel module loading includes error handling for modules that might be built into the kernel
|
||||
- DNS configuration is specifically adjusted for systemd-resolved in Ubuntu 24.04
|
||||
|
||||
## Validation
|
||||
|
||||
After deployment, verify your cluster:
|
||||
|
||||
```bash
|
||||
ssh <your-user>@<controlplane-ip>
|
||||
kubectl get nodes
|
||||
```
|
||||
|
||||
You should see all nodes in the `Ready` state:
|
||||
|
||||
```
|
||||
NAME STATUS ROLES AGE VERSION
|
||||
controlplane Ready control-plane 10m v1.28.x
|
||||
node01 Ready <none> 8m v1.28.x
|
||||
node02 Ready <none> 8m v1.28.x
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
If nodes don't join the cluster or show issues:
|
||||
|
||||
1. Check connectivity between nodes: `ping <node-ip>`
|
||||
2. Verify firewall allows required ports (6443, 10250)
|
||||
3. Review logs on problematic nodes: `journalctl -xeu kubelet`
|
||||
4. Verify DNS resolution: `nslookup controlplane`
|
||||
|
||||
## Cleanup
|
||||
|
||||
To reset a node and remove Kubernetes:
|
||||
|
||||
```bash
|
||||
sudo kubeadm reset
|
||||
sudo apt-get purge -y kubeadm kubectl kubelet kubernetes-cni
|
||||
sudo apt-get autoremove -y
|
||||
sudo rm -rf ~/.kube /etc/kubernetes /var/lib/kubelet /var/lib/etcd
|
||||
```
|
||||
@@ -0,0 +1,22 @@
|
||||
---
|
||||
all:
|
||||
children:
|
||||
k8s_cluster:
|
||||
children:
|
||||
control_plane:
|
||||
hosts:
|
||||
controlplane:
|
||||
ansible_host: 10.203.146.213
|
||||
workers:
|
||||
hosts:
|
||||
node01:
|
||||
ansible_host: 10.203.146.194
|
||||
node02:
|
||||
ansible_host: 10.203.146.61
|
||||
node03:
|
||||
ansible_host: 10.203.146.193
|
||||
|
||||
vars:
|
||||
ansible_user: ubuntu
|
||||
ansible_become: yes
|
||||
ansible_ssh_private_key_file: ~/.ssh/id_rsa
|
||||
@@ -0,0 +1,45 @@
|
||||
---
|
||||
- name: Common configuration for all nodes
|
||||
hosts: k8s_cluster
|
||||
become: yes
|
||||
tags:
|
||||
- common
|
||||
tasks:
|
||||
- name: Include hosts setup
|
||||
include_tasks: tasks/setup-hosts.yml
|
||||
|
||||
- name: Include DNS setup
|
||||
include_tasks: tasks/update-dns.yml
|
||||
|
||||
- name: Include SSH setup
|
||||
include_tasks: tasks/ssh.yml
|
||||
|
||||
- name: Include kernel setup
|
||||
include_tasks: tasks/setup-kernel.yml
|
||||
|
||||
- name: Kubernetes setup for all nodes
|
||||
hosts: k8s_cluster
|
||||
become: yes
|
||||
tags:
|
||||
- k8s-setup
|
||||
tasks:
|
||||
- name: Include Kubernetes node setup
|
||||
include_tasks: tasks/node-setup.yml
|
||||
|
||||
- name: Initialize Kubernetes control plane
|
||||
hosts: control_plane
|
||||
become: yes
|
||||
tags:
|
||||
- control-plane
|
||||
tasks:
|
||||
- name: Initialize control plane
|
||||
include_tasks: tasks/control-plane-init.yml
|
||||
|
||||
- name: Join worker nodes to cluster
|
||||
hosts: workers
|
||||
become: yes
|
||||
tags:
|
||||
- workers
|
||||
tasks:
|
||||
- name: Join worker nodes
|
||||
include_tasks: tasks/worker-join.yml
|
||||
@@ -0,0 +1,98 @@
|
||||
---
|
||||
# tasks/control-plane-init.yml
|
||||
- name: Set Kubernetes network CIDRs for Calico
|
||||
set_fact:
|
||||
pod_cidr: "192.168.0.0/16" # Calico default CIDR
|
||||
service_cidr: "10.96.0.0/16"
|
||||
|
||||
- name: Initialize Kubernetes control plane
|
||||
command: >
|
||||
kubeadm init
|
||||
--pod-network-cidr={{ pod_cidr }}
|
||||
--service-cidr={{ service_cidr }}
|
||||
--apiserver-advertise-address={{ ansible_default_ipv4.address }}
|
||||
register: kubeadm_init
|
||||
args:
|
||||
creates: /etc/kubernetes/admin.conf
|
||||
|
||||
- name: Create .kube directory for current user
|
||||
file:
|
||||
path: "/home/{{ ansible_user }}/.kube"
|
||||
state: directory
|
||||
owner: "{{ ansible_user }}"
|
||||
group: "{{ ansible_user }}"
|
||||
mode: '0755'
|
||||
|
||||
- name: Copy Kubernetes admin.conf to user's .kube/config
|
||||
copy:
|
||||
src: /etc/kubernetes/admin.conf
|
||||
dest: "/home/{{ ansible_user }}/.kube/config"
|
||||
remote_src: yes
|
||||
owner: "{{ ansible_user }}"
|
||||
group: "{{ ansible_user }}"
|
||||
mode: '0644'
|
||||
|
||||
- name: Install Calico Operator
|
||||
shell: kubectl --kubeconfig=/etc/kubernetes/admin.conf create -f https://raw.githubusercontent.com/projectcalico/calico/v3.29.1/manifests/tigera-operator.yaml
|
||||
register: calico_operator_result
|
||||
changed_when: "'created' in calico_operator_result.stdout"
|
||||
failed_when:
|
||||
- calico_operator_result.rc != 0
|
||||
- "'already exists' not in calico_operator_result.stderr"
|
||||
|
||||
- name: Install Calico Custom Resources
|
||||
shell: kubectl --kubeconfig=/etc/kubernetes/admin.conf create -f https://raw.githubusercontent.com/projectcalico/calico/v3.29.1/manifests/custom-resources.yaml
|
||||
register: calico_resources_result
|
||||
changed_when: "'created' in calico_resources_result.stdout"
|
||||
failed_when:
|
||||
- calico_resources_result.rc != 0
|
||||
- "'already exists' not in calico_resources_result.stderr"
|
||||
|
||||
- name: Wait for calico-system namespace to be created
|
||||
shell: kubectl --kubeconfig=/etc/kubernetes/admin.conf get namespace | grep calico-system
|
||||
register: calico_ns
|
||||
until: calico_ns.rc == 0
|
||||
retries: 30
|
||||
delay: 10
|
||||
changed_when: false
|
||||
|
||||
- name: Wait for Calico pods to be ready
|
||||
shell: |
|
||||
# Give pods some time to be created
|
||||
sleep 30
|
||||
kubectl --kubeconfig=/etc/kubernetes/admin.conf -n calico-system wait --for=condition=ready pods --all --timeout=300s
|
||||
register: calico_pods_ready
|
||||
changed_when: false
|
||||
failed_when: false # Don't fail if timeout occurs, we'll verify the node status later
|
||||
|
||||
- name: Wait for CoreDNS to be ready
|
||||
shell: kubectl --kubeconfig=/etc/kubernetes/admin.conf -n kube-system wait --for=condition=ready pods -l k8s-app=kube-dns --timeout=120s
|
||||
register: coredns_ready
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
|
||||
- name: Wait for node to be ready
|
||||
shell: kubectl --kubeconfig=/etc/kubernetes/admin.conf wait --for=condition=ready nodes --all --timeout=120s
|
||||
register: node_ready
|
||||
changed_when: false
|
||||
|
||||
- name: Remove taint from control plane node
|
||||
shell: kubectl --kubeconfig=/etc/kubernetes/admin.conf taint nodes --all node-role.kubernetes.io/control-plane-
|
||||
register: taint_result
|
||||
changed_when: "'untainted' in taint_result.stdout"
|
||||
failed_when: false # This might fail if the taint doesn't exist, which is fine
|
||||
|
||||
- name: Remove load balancer label from all nodes
|
||||
shell: kubectl --kubeconfig=/etc/kubernetes/admin.conf label nodes --all node.kubernetes.io/exclude-from-external-load-balancers-
|
||||
register: label_result
|
||||
changed_when: "'labeled' in label_result.stdout"
|
||||
failed_when: false # This might fail if the label doesn't exist, which is fine
|
||||
|
||||
- name: Get join command
|
||||
command: kubeadm token create --print-join-command
|
||||
register: join_command
|
||||
changed_when: false
|
||||
|
||||
- name: Store join command
|
||||
set_fact:
|
||||
join_command: "{{ join_command.stdout }}"
|
||||
@@ -0,0 +1,169 @@
|
||||
---
|
||||
# tasks/node-setup.yml
|
||||
- name: Update apt cache
|
||||
apt:
|
||||
update_cache: yes
|
||||
|
||||
- name: Install apt transport packages
|
||||
apt:
|
||||
name:
|
||||
- apt-transport-https
|
||||
- ca-certificates
|
||||
- curl
|
||||
- gnupg
|
||||
state: present
|
||||
|
||||
# Add Docker GPG key
|
||||
- name: Add Docker GPG key
|
||||
apt_key:
|
||||
url: https://download.docker.com/linux/ubuntu/gpg
|
||||
state: present
|
||||
|
||||
- name: Add Docker repository
|
||||
apt_repository:
|
||||
repo: "deb [arch=amd64] https://download.docker.com/linux/ubuntu {{ ansible_distribution_release }} stable"
|
||||
state: present
|
||||
filename: docker
|
||||
|
||||
- name: Install containerd from Docker repository
|
||||
apt:
|
||||
name: containerd.io
|
||||
state: present
|
||||
update_cache: yes
|
||||
|
||||
- name: Create containerd directory
|
||||
file:
|
||||
path: /etc/containerd
|
||||
state: directory
|
||||
mode: '0755'
|
||||
|
||||
- name: Check if containerd config command works
|
||||
command: which containerd
|
||||
register: containerd_check
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
|
||||
- name: Generate default containerd config (if containerd exists)
|
||||
command: containerd config default
|
||||
register: containerd_config
|
||||
changed_when: false
|
||||
when: containerd_check.rc == 0
|
||||
|
||||
- name: Create a basic containerd config (if command failed)
|
||||
copy:
|
||||
content: |
|
||||
version = 2
|
||||
|
||||
[plugins]
|
||||
[plugins."io.containerd.grpc.v1.cri"]
|
||||
[plugins."io.containerd.grpc.v1.cri".containerd]
|
||||
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes]
|
||||
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
|
||||
runtime_type = "io.containerd.runc.v2"
|
||||
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
|
||||
SystemdCgroup = true
|
||||
dest: /etc/containerd/config.toml
|
||||
owner: root
|
||||
group: root
|
||||
mode: '0644'
|
||||
when: containerd_check.rc != 0
|
||||
|
||||
- name: Write containerd config from generated output
|
||||
copy:
|
||||
content: "{{ containerd_config.stdout }}"
|
||||
dest: /etc/containerd/config.toml
|
||||
owner: root
|
||||
group: root
|
||||
mode: '0644'
|
||||
when: containerd_check.rc == 0 and containerd_config.stdout is defined and containerd_config.stdout != ""
|
||||
|
||||
- name: Enable SystemdCgroup in containerd config
|
||||
replace:
|
||||
path: /etc/containerd/config.toml
|
||||
regexp: 'SystemdCgroup = false'
|
||||
replace: 'SystemdCgroup = true'
|
||||
when: containerd_check.rc == 0
|
||||
|
||||
- name: Restart containerd
|
||||
systemd:
|
||||
name: containerd
|
||||
state: restarted
|
||||
enabled: yes
|
||||
|
||||
- name: Get latest Kubernetes version
|
||||
shell: curl -L -s https://dl.k8s.io/release/stable.txt | awk 'BEGIN { FS="." } { printf "%s.%s", $1, $2 }'
|
||||
register: kube_latest
|
||||
changed_when: false
|
||||
|
||||
- name: Create keyrings directory
|
||||
file:
|
||||
path: /etc/apt/keyrings
|
||||
state: directory
|
||||
mode: '0755'
|
||||
|
||||
- name: Download Kubernetes apt key
|
||||
get_url:
|
||||
url: "https://pkgs.k8s.io/core:/stable:/{{ kube_latest.stdout }}/deb/Release.key"
|
||||
dest: /tmp/kubernetes-release.key
|
||||
mode: '0644'
|
||||
|
||||
- name: Add Kubernetes apt key
|
||||
shell: cat /tmp/kubernetes-release.key | gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
|
||||
args:
|
||||
creates: /etc/apt/keyrings/kubernetes-apt-keyring.gpg
|
||||
|
||||
- name: Add Kubernetes apt repository
|
||||
copy:
|
||||
content: "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/{{ kube_latest.stdout }}/deb/ /"
|
||||
dest: /etc/apt/sources.list.d/kubernetes.list
|
||||
owner: root
|
||||
group: root
|
||||
mode: '0644'
|
||||
|
||||
- name: Update apt cache
|
||||
apt:
|
||||
update_cache: yes
|
||||
|
||||
- name: Install Kubernetes components
|
||||
apt:
|
||||
name:
|
||||
- kubelet
|
||||
- kubeadm
|
||||
- kubectl
|
||||
state: present
|
||||
|
||||
- name: Hold Kubernetes packages
|
||||
dpkg_selections:
|
||||
name: "{{ item }}"
|
||||
selection: hold
|
||||
loop:
|
||||
- kubelet
|
||||
- kubeadm
|
||||
- kubectl
|
||||
|
||||
- name: Check if crictl is installed
|
||||
command: which crictl
|
||||
register: crictl_check
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
|
||||
- name: Configure crictl
|
||||
shell: |
|
||||
crictl config \
|
||||
--set runtime-endpoint=unix:///run/containerd/containerd.sock \
|
||||
--set image-endpoint=unix:///run/containerd/containerd.sock
|
||||
changed_when: false
|
||||
when: crictl_check.rc == 0
|
||||
|
||||
- name: Get primary network interface IP
|
||||
shell: ip -4 addr show {{ ansible_default_ipv4.interface }} | grep "inet" | head -1 | awk '{print $2}' | cut -d/ -f1
|
||||
register: node_ip
|
||||
changed_when: false
|
||||
|
||||
- name: Configure kubelet node IP
|
||||
copy:
|
||||
content: "KUBELET_EXTRA_ARGS='--node-ip {{ node_ip.stdout }}'"
|
||||
dest: /etc/default/kubelet
|
||||
owner: root
|
||||
group: root
|
||||
mode: '0644'
|
||||
@@ -0,0 +1,33 @@
|
||||
---
|
||||
- name: Remove unwanted entries from /etc/hosts
|
||||
lineinfile:
|
||||
path: /etc/hosts
|
||||
regexp: "{{ item }}"
|
||||
state: absent
|
||||
loop:
|
||||
- "^.*ubuntu-jammy.*$"
|
||||
- "^.*{{ ansible_hostname }}.*$"
|
||||
|
||||
- name: Get primary IP address
|
||||
shell: "ip route | grep default | grep -Pv '10\\.\\d+\\.\\d+\\.\\d+' | awk '{ print $9 }'"
|
||||
register: primary_ip
|
||||
changed_when: false
|
||||
|
||||
- name: Set PRIMARY_IP environment variable
|
||||
lineinfile:
|
||||
path: /etc/environment
|
||||
line: "PRIMARY_IP={{ primary_ip.stdout }}"
|
||||
state: present
|
||||
|
||||
- name: Add controlplane entry to /etc/hosts
|
||||
lineinfile:
|
||||
path: /etc/hosts
|
||||
line: "{{ hostvars['controlplane']['ansible_host'] }} controlplane"
|
||||
state: present
|
||||
|
||||
- name: Add worker nodes to /etc/hosts
|
||||
lineinfile:
|
||||
path: /etc/hosts
|
||||
line: "{{ hostvars[item]['ansible_host'] }} {{ item }}"
|
||||
state: present
|
||||
loop: "{{ groups['workers'] }}"
|
||||
@@ -0,0 +1,44 @@
|
||||
---
|
||||
# tasks/setup-kernel.yml
|
||||
- name: Create Kubernetes kernel modules config file
|
||||
copy:
|
||||
content: |
|
||||
overlay
|
||||
br_netfilter
|
||||
dest: /etc/modules-load.d/k8s.conf
|
||||
owner: root
|
||||
group: root
|
||||
mode: '0644'
|
||||
|
||||
- name: Attempt to load overlay module
|
||||
shell: modprobe overlay || true
|
||||
changed_when: false
|
||||
|
||||
- name: Attempt to load br_netfilter module
|
||||
shell: modprobe br_netfilter || true
|
||||
changed_when: false
|
||||
|
||||
- name: Create Kubernetes sysctl config file
|
||||
copy:
|
||||
content: |
|
||||
net.bridge.bridge-nf-call-iptables = 1
|
||||
net.bridge.bridge-nf-call-ip6tables = 1
|
||||
net.ipv4.ip_forward = 1
|
||||
dest: /etc/sysctl.d/k8s.conf
|
||||
owner: root
|
||||
group: root
|
||||
mode: '0644'
|
||||
register: sysctl_conf
|
||||
|
||||
- name: Apply sysctl changes
|
||||
command: sysctl --system
|
||||
when: sysctl_conf.changed
|
||||
|
||||
- name: Check loaded modules
|
||||
shell: "lsmod | grep -E 'overlay|br_netfilter' || echo 'Modules may be built into kernel'"
|
||||
register: modules_check
|
||||
changed_when: false
|
||||
|
||||
- name: Display module status
|
||||
debug:
|
||||
var: modules_check.stdout_lines
|
||||
@@ -0,0 +1,18 @@
|
||||
---
|
||||
# tasks/ssh.yml
|
||||
- name: Enable password authentication in sshd_config
|
||||
replace:
|
||||
path: /etc/ssh/sshd_config
|
||||
regexp: '#PasswordAuthentication'
|
||||
replace: 'PasswordAuthentication'
|
||||
|
||||
- name: Enable KbdInteractiveAuthentication in sshd_config
|
||||
replace:
|
||||
path: /etc/ssh/sshd_config
|
||||
regexp: 'KbdInteractiveAuthentication no'
|
||||
replace: 'KbdInteractiveAuthentication yes'
|
||||
|
||||
- name: Restart SSH service
|
||||
systemd:
|
||||
name: ssh
|
||||
state: restarted
|
||||
@@ -0,0 +1,18 @@
|
||||
---
|
||||
- name: Ensure systemd-resolved is running
|
||||
systemd:
|
||||
name: systemd-resolved
|
||||
state: started
|
||||
enabled: yes
|
||||
|
||||
- name: Configure systemd-resolved to use Google DNS
|
||||
lineinfile:
|
||||
path: /etc/systemd/resolved.conf
|
||||
regexp: '^#?DNS='
|
||||
line: 'DNS=8.8.8.8'
|
||||
state: present
|
||||
|
||||
- name: Restart systemd-resolved service
|
||||
systemd:
|
||||
name: systemd-resolved
|
||||
state: restarted
|
||||
@@ -0,0 +1,13 @@
|
||||
---
|
||||
# tasks/worker-join.yml
|
||||
- name: Get join command from control plane
|
||||
delegate_to: controlplane
|
||||
command: kubeadm token create --print-join-command
|
||||
register: join_command_result
|
||||
changed_when: false
|
||||
run_once: true
|
||||
|
||||
- name: Join worker node to the cluster
|
||||
command: "{{ join_command_result.stdout }}"
|
||||
args:
|
||||
creates: /etc/kubernetes/kubelet.conf
|
||||
@@ -0,0 +1,246 @@
|
||||
# Install Multipass
|
||||
|
||||
```bash
|
||||
sudo apt update
|
||||
sudo apt upgrade
|
||||
sudo snap install multipass
|
||||
```
|
||||
|
||||
To fast-track to build multiple instances from the cloudinit.yaml file run e.g.
|
||||
```bash
|
||||
./launch-instances.sh cp1 worker1 worker2 worker3 worker4
|
||||
```
|
||||
Then skip ahead to `# Install and Configure Network Plugin 'Calico'` further down in this file
|
||||
|
||||
# Launch
|
||||
## Launch basic instance
|
||||
```bash
|
||||
multipass launch --name worker-01
|
||||
```
|
||||
|
||||
## Launch instance with defined resources
|
||||
```bash
|
||||
multipass launch --name worker-02 --disk 20G --memory 8G --cpus 2
|
||||
```
|
||||
|
||||
## Launch instance with defined resources and cloudinit
|
||||
create a yaml file which defines the intial state of the ubuntu instance
|
||||
for example `cloudinit.yaml`
|
||||
|
||||
and run a launch command like this after you have created the subsequent file
|
||||
```bash
|
||||
multipass launch --name cp1 --cloud-init cloudinit.yaml --disk 20G --memory 8G --cpus 2
|
||||
```
|
||||
|
||||
```yaml
|
||||
#cloud-config
|
||||
package_update: true
|
||||
package_upgrade: true
|
||||
|
||||
write_files:
|
||||
- path: /etc/modules-load.d/containerd.conf
|
||||
content: |
|
||||
overlay
|
||||
br_netfilter
|
||||
- path: /etc/sysctl.d/99-kubernetes-cri.conf
|
||||
content: |
|
||||
net.bridge.bridge-nf-call-iptables = 1
|
||||
net.bridge.bridge-nf-call-ip6tables = 1
|
||||
net.ipv4.ip_forward = 1
|
||||
|
||||
runcmd:
|
||||
# Disable swap
|
||||
- swapoff -a
|
||||
- sed -i '/swap/d' /etc/fstab
|
||||
|
||||
# Load necessary modules
|
||||
- modprobe overlay
|
||||
- modprobe br_netfilter
|
||||
|
||||
# Apply sysctl params
|
||||
- sysctl --system
|
||||
|
||||
# Remove conflicting packages
|
||||
- for pkg in docker.io docker-doc docker-compose docker-compose-v2 podman-docker containerd runc; do apt-get remove -y $pkg; done
|
||||
|
||||
# Install required packages
|
||||
- apt-get install -y ca-certificates curl gpg apt-transport-https
|
||||
|
||||
# Add Docker GPG key
|
||||
- mkdir -p -m 0755 /etc/apt/keyrings
|
||||
- curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
|
||||
- chmod a+r /etc/apt/keyrings/docker.asc
|
||||
|
||||
# Add Docker repository
|
||||
- echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo ${UBUNTU_CODENAME:-$VERSION_CODENAME}) stable" | tee /etc/apt/sources.list.d/docker.list
|
||||
|
||||
# Update package lists
|
||||
- apt-get update
|
||||
|
||||
# Install containerd
|
||||
- apt-get install -y containerd.io
|
||||
|
||||
# Configure containerd
|
||||
- mkdir -p /etc/containerd
|
||||
- containerd config default | tee /etc/containerd/config.toml
|
||||
- sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml
|
||||
- systemctl restart containerd
|
||||
- systemctl enable containerd
|
||||
|
||||
# Add Kubernetes GPG key
|
||||
- curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.32/deb/Release.key | gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
|
||||
|
||||
# Add Kubernetes repository
|
||||
- echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.32/deb/ /" | tee /etc/apt/sources.list.d/kubernetes.list
|
||||
|
||||
# Update package lists
|
||||
- apt-get update
|
||||
|
||||
# Install kubelet, kubeadm, and kubectl
|
||||
- apt-get install -y kubelet kubeadm kubectl
|
||||
- apt-mark hold kubelet kubeadm kubectl
|
||||
|
||||
# Enable kubelet service
|
||||
- systemctl enable --now kubelet
|
||||
|
||||
# pull kubeadm related image
|
||||
- sudo kubeadm config images pull
|
||||
```
|
||||
|
||||
When the control plane instance is running, you have to install a network plugin
|
||||
from a third party provider.
|
||||
|
||||
# Install and Configure Network Plugin 'Calico'
|
||||
|
||||
Shell into the control plane instance with `multipass shell <instance-name>` and run
|
||||
|
||||
```bash
|
||||
sudo kubeadm init --pod-network-cidr=192.168.0.0/16
|
||||
mkdir -p $HOME/.kube
|
||||
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
|
||||
sudo chown $(id -u):$(id -g) $HOME/.kube/config
|
||||
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.29.1/manifests/tigera-operator.yaml
|
||||
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.29.1/manifests/custom-resources.yaml
|
||||
watch kubectl get pods -n calico-system
|
||||
```
|
||||
Wait until all the pods are up and running.
|
||||
|
||||
Afterwards run
|
||||
```bash
|
||||
kubectl taint nodes --all node-role.kubernetes.io/control-plane-
|
||||
```
|
||||
It should return something like
|
||||
```bash
|
||||
node/<your-hostname> untainted
|
||||
```
|
||||
|
||||
Then taint the control plane to not get any schedules, only the workers
|
||||
```bash
|
||||
kubectl taint nodes <control-plane-name-or-ip> node-role.kubernetes.io/control-plane=:NoSchedule
|
||||
```
|
||||
|
||||
Confirm that you have a node running in your cluster
|
||||
```bash
|
||||
kubectl get nodes -o wide
|
||||
```
|
||||
It should return something like this
|
||||
```bash
|
||||
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
|
||||
<your-hostname> Ready master 52m v1.12.2 10.128.0.28 <none> Ubuntu 18.04.1 LTS 4.15.0-1023-gcp docker://18.6.1
|
||||
```
|
||||
Perhaps remove the load balancer label from the list of backend servers
|
||||
```bash
|
||||
kubectl label nodes --all node.kubernetes.io/exclude-from-external-load-balancers-
|
||||
```
|
||||
|
||||
# Adding Linux Worker Nodes
|
||||
Run the same `cloudinit.yaml` file as the control plane instance.
|
||||
For example
|
||||
```bash
|
||||
multipass launch --name worker1 --cloud-init cloudinit.yaml --disk 20G --memory 8G --cpus 2
|
||||
```
|
||||
When the instance is ready, shell into the instance with `multipass shell worker1` and run
|
||||
```bash
|
||||
sudo kubeadm join --token <token> <control-plane-host>:<control-plane-port> --discovery-token-ca-cert-hash sha256:<hash>
|
||||
```
|
||||
|
||||
[Important]
|
||||
To get the exact information for the join run
|
||||
```bash
|
||||
multipass shell <controlplane-name>
|
||||
```
|
||||
When inside, run
|
||||
```bash
|
||||
sudo kubeadm token create --print-join-command
|
||||
```
|
||||
|
||||
The information/token was generated with the kubeadm init command.
|
||||
If you need to recover the information, shell into `multipass shell cp1`
|
||||
and run
|
||||
```bash
|
||||
# Run this on a control plane node
|
||||
sudo kubeadm token list
|
||||
```
|
||||
|
||||
If you do not want to shell into the control plane you should be able to run (from dev pc)
|
||||
```bash
|
||||
multipass exec cp1 sudo kubeadm token list
|
||||
```
|
||||
which should return something like this
|
||||
```bash
|
||||
TOKEN TTL EXPIRES USAGES DESCRIPTION EXTRA GROUPS
|
||||
evv51i.5p35yitsiva6sjer 23h 2025-02-05T20:52:23Z authentication,signing The default bootstrap token generated by 'kubeadm init'. system:bootstrappers:kubeadm:default-node-token
|
||||
```
|
||||
|
||||
By default, node join tokens expire after 24 hours.
|
||||
If you are joining a node to the cluster after the current token has expired,
|
||||
you can create a new token by running the following command on the control plane node:
|
||||
```bash
|
||||
# Run this on a control plane node
|
||||
sudo kubeadm token create
|
||||
```
|
||||
The output is similar to this:
|
||||
```bash
|
||||
5didvk.d09sbcov8ph2amjw
|
||||
```
|
||||
|
||||
If you don't have the value of `--discovery-token-ca-cert-hash`,
|
||||
you can get it by running the following commands on the control plane node:
|
||||
|
||||
```bash
|
||||
# Run this on a control plane node
|
||||
sudo cat /etc/kubernetes/pki/ca.crt | openssl x509 -pubkey | openssl rsa -pubin -outform der 2>/dev/null | \
|
||||
openssl dgst -sha256 -hex | sed 's/^.* //'
|
||||
```
|
||||
|
||||
The output is similar to:
|
||||
```bash
|
||||
8cb2de97839780a412b93877f8507ad6c94f73add17d5d7058e91741c9d5ec78
|
||||
```
|
||||
|
||||
The output of the kubeadm join command should look something like:
|
||||
```bash
|
||||
[preflight] Running pre-flight checks
|
||||
|
||||
... (log output of join workflow) ...
|
||||
|
||||
Node join complete:
|
||||
* Certificate signing request sent to control-plane and response
|
||||
received.
|
||||
* Kubelet informed of new secure connection details.
|
||||
|
||||
|
||||
Run 'kubectl get nodes' on control-plane to see this machine join.
|
||||
```
|
||||
A few seconds later, you should notice this node in the output from kubectl get nodes.
|
||||
(for example, run kubectl on a control plane node).
|
||||
|
||||
[Note]
|
||||
As the cluster nodes are usually initialized sequentially,
|
||||
the CoreDNS Pods are likely to all run on the first control plane node.
|
||||
To provide higher availability,
|
||||
please rebalance the CoreDNS Pods with
|
||||
```bash
|
||||
kubectl -n kube-system rollout restart deployment coredns
|
||||
```
|
||||
after at least one new node is joined.
|
||||
@@ -0,0 +1,170 @@
|
||||
---
|
||||
- name: Setup Kubernetes Node with Docker
|
||||
hosts: k8s_nodes
|
||||
become: true
|
||||
|
||||
tasks:
|
||||
- name: Update package cache
|
||||
apt:
|
||||
update_cache: yes
|
||||
|
||||
- name: Upgrade all packages
|
||||
apt:
|
||||
upgrade: yes
|
||||
|
||||
- name: Create kernel module configuration
|
||||
copy:
|
||||
dest: /etc/modules-load.d/k8s.conf
|
||||
content: |
|
||||
overlay
|
||||
br_netfilter
|
||||
mode: '0644'
|
||||
|
||||
- name: Create sysctl configuration
|
||||
copy:
|
||||
dest: /etc/sysctl.d/99-kubernetes-cri.conf
|
||||
content: |
|
||||
net.bridge.bridge-nf-call-iptables = 1
|
||||
net.bridge.bridge-nf-call-ip6tables = 1
|
||||
net.ipv4.ip_forward = 1
|
||||
mode: '0644'
|
||||
|
||||
- name: Disable swap
|
||||
block:
|
||||
- name: Disable swap immediately
|
||||
command: swapoff -a
|
||||
changed_when: false
|
||||
- name: Remove swap from fstab
|
||||
replace:
|
||||
path: /etc/fstab
|
||||
regexp: '^([^#].*?\sswap\s+sw\s+.*)$'
|
||||
replace: ''
|
||||
|
||||
- name: Load kernel modules
|
||||
command: "modprobe {{ item }}"
|
||||
loop:
|
||||
- overlay
|
||||
- br_netfilter
|
||||
changed_when: false
|
||||
|
||||
- name: Apply sysctl params
|
||||
command: sysctl --system
|
||||
changed_when: false
|
||||
|
||||
- name: Remove conflicting packages
|
||||
apt:
|
||||
name:
|
||||
- docker
|
||||
- docker.io
|
||||
- docker-doc
|
||||
- docker-compose
|
||||
- docker-compose-v2
|
||||
- podman-docker
|
||||
- containerd
|
||||
- runc
|
||||
state: absent
|
||||
|
||||
- name: Install required packages
|
||||
apt:
|
||||
name:
|
||||
- ca-certificates
|
||||
- curl
|
||||
- gnupg
|
||||
state: present
|
||||
|
||||
- name: Create keyrings directory
|
||||
file:
|
||||
path: /etc/apt/keyrings
|
||||
state: directory
|
||||
mode: '0755'
|
||||
|
||||
- name: Add Docker GPG key
|
||||
shell: |
|
||||
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg
|
||||
chmod a+r /etc/apt/keyrings/docker.gpg
|
||||
|
||||
- name: Add Docker repository
|
||||
shell: |
|
||||
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo $VERSION_CODENAME) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null
|
||||
|
||||
- name: Update package lists
|
||||
apt:
|
||||
update_cache: yes
|
||||
|
||||
- name: Install Docker packages
|
||||
apt:
|
||||
name:
|
||||
- docker-ce
|
||||
- docker-ce-cli
|
||||
- containerd.io
|
||||
- docker-buildx-plugin
|
||||
- docker-compose-plugin
|
||||
state: present
|
||||
|
||||
- name: Create Docker daemon configuration
|
||||
copy:
|
||||
dest: /etc/docker/daemon.json
|
||||
content: |
|
||||
{
|
||||
"exec-opts": ["native.cgroupdriver=systemd"],
|
||||
"log-driver": "json-file",
|
||||
"log-opts": {
|
||||
"max-size": "100m"
|
||||
},
|
||||
"storage-driver": "overlay2"
|
||||
}
|
||||
mode: '0644'
|
||||
create: yes
|
||||
directory_mode: '0755'
|
||||
|
||||
- name: Create Docker service directory
|
||||
file:
|
||||
path: /etc/systemd/system/docker.service.d
|
||||
state: directory
|
||||
mode: '0755'
|
||||
|
||||
- name: Restart Docker
|
||||
systemd:
|
||||
name: docker
|
||||
state: restarted
|
||||
daemon_reload: yes
|
||||
enabled: yes
|
||||
|
||||
- name: Add Kubernetes GPG key
|
||||
shell: |
|
||||
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.32/deb/Release.key | gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
|
||||
|
||||
- name: Add Kubernetes repository
|
||||
shell: |
|
||||
echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.32/deb/ /" | tee /etc/apt/sources.list.d/kubernetes.list
|
||||
|
||||
- name: Update package lists again
|
||||
apt:
|
||||
update_cache: yes
|
||||
|
||||
- name: Install Kubernetes packages
|
||||
apt:
|
||||
name:
|
||||
- kubelet
|
||||
- kubeadm
|
||||
- kubectl
|
||||
state: present
|
||||
|
||||
- name: Hold Kubernetes packages
|
||||
dpkg_selections:
|
||||
name: "{{ item }}"
|
||||
selection: hold
|
||||
loop:
|
||||
- kubelet
|
||||
- kubeadm
|
||||
- kubectl
|
||||
|
||||
- name: Enable kubelet service
|
||||
systemd:
|
||||
name: kubelet
|
||||
state: started
|
||||
enabled: yes
|
||||
|
||||
- name: Pull kubeadm images
|
||||
command: kubeadm config images pull
|
||||
changed_when: false
|
||||
@@ -0,0 +1,73 @@
|
||||
#cloud-config
|
||||
package_update: true
|
||||
package_upgrade: true
|
||||
|
||||
write_files:
|
||||
- path: /etc/modules-load.d/containerd.conf
|
||||
content: |
|
||||
overlay
|
||||
br_netfilter
|
||||
- path: /etc/sysctl.d/99-kubernetes-cri.conf
|
||||
content: |
|
||||
net.bridge.bridge-nf-call-iptables = 1
|
||||
net.bridge.bridge-nf-call-ip6tables = 1
|
||||
net.ipv4.ip_forward = 1
|
||||
|
||||
runcmd:
|
||||
# Disable swap
|
||||
- swapoff -a
|
||||
- sed -i '/swap/d' /etc/fstab
|
||||
|
||||
# Load necessary modules
|
||||
- modprobe overlay
|
||||
- modprobe br_netfilter
|
||||
|
||||
# Apply sysctl params
|
||||
- sysctl --system
|
||||
|
||||
# Remove conflicting packages
|
||||
- for pkg in docker.io docker-doc docker-compose docker-compose-v2 podman-docker containerd runc; do apt-get remove -y $pkg; done
|
||||
|
||||
# Install required packages
|
||||
- apt-get install -y ca-certificates curl gpg apt-transport-https
|
||||
|
||||
# Add Docker GPG key
|
||||
- mkdir -p -m 0755 /etc/apt/keyrings
|
||||
- curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
|
||||
- chmod a+r /etc/apt/keyrings/docker.asc
|
||||
|
||||
# Add Docker repository
|
||||
- echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo ${UBUNTU_CODENAME:-$VERSION_CODENAME}) stable" | tee /etc/apt/sources.list.d/docker.list
|
||||
|
||||
# Update package lists
|
||||
- apt-get update
|
||||
|
||||
# Install containerd
|
||||
- apt-get install -y containerd.io
|
||||
|
||||
# Configure containerd
|
||||
- mkdir -p /etc/containerd
|
||||
- containerd config default | tee /etc/containerd/config.toml
|
||||
- sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml
|
||||
- systemctl restart containerd
|
||||
- systemctl enable containerd
|
||||
|
||||
# Add Kubernetes GPG key
|
||||
- curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.32/deb/Release.key | gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
|
||||
|
||||
# Add Kubernetes repository
|
||||
- echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.32/deb/ /" | tee /etc/apt/sources.list.d/kubernetes.list
|
||||
|
||||
# Update package lists
|
||||
- apt-get update
|
||||
|
||||
# Install kubelet, kubeadm, and kubectl
|
||||
- apt-get install -y kubelet kubeadm kubectl
|
||||
- apt-mark hold kubelet kubeadm kubectl
|
||||
|
||||
# Enable kubelet service
|
||||
- systemctl enable --now kubelet
|
||||
|
||||
# pull kubeadm related image
|
||||
- sudo kubeadm config images pull
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
[k8s_nodes]
|
||||
node1 ansible_host=10.203.146.243
|
||||
|
||||
[k8s_nodes:vars]
|
||||
ansible_user=ubuntu
|
||||
ansible_ssh_private_key_file=~/.ssh/id_rsa
|
||||
@@ -0,0 +1,8 @@
|
||||
[k8s_nodes]
|
||||
node1 ansible_host=10.203.146.216
|
||||
|
||||
[k8s_nodes:vars]
|
||||
ansible_user=ubuntu
|
||||
ansible_ssh_private_key_file=~/.ssh/id_rsa
|
||||
ansible_become=true
|
||||
|
||||
+13
@@ -0,0 +1,13 @@
|
||||
#!/bin/bash
|
||||
|
||||
if [ "$#" -eq 0 ]; then
|
||||
echo "Usage: $0 instance_name [instance_name2 ... instance_nameN]"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Iterate over all provided instance names
|
||||
for INSTANCE in "$@"; do
|
||||
echo "Launching instance: ${INSTANCE}..."
|
||||
multipass launch --name "${INSTANCE}" --cloud-init cloudinit.yaml --disk 20G --memory 8G --cpus 2
|
||||
done
|
||||
|
||||
@@ -0,0 +1,36 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
labels:
|
||||
app: asyncapi
|
||||
name: asyncapi-deployment
|
||||
namespace: asyncapi
|
||||
spec:
|
||||
replicas: 2
|
||||
selector:
|
||||
matchLabels:
|
||||
app: asyncapi
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: asyncapi
|
||||
spec:
|
||||
containers:
|
||||
- name: asyncapi
|
||||
image: iitaks.azurecr.io/asyncapi:1.2
|
||||
ports:
|
||||
- containerPort: 80
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: asyncapi-service
|
||||
namespace: asyncapi
|
||||
spec:
|
||||
selector:
|
||||
app: asyncapi
|
||||
type: NodePort
|
||||
ports:
|
||||
- protocol: TCP
|
||||
port: 80
|
||||
targetPort: 80
|
||||
+97
@@ -0,0 +1,97 @@
|
||||
# Installation
|
||||
```bash
|
||||
# Add Docker's official GPG key:
|
||||
sudo apt-get update
|
||||
sudo apt-get install ca-certificates curl
|
||||
sudo install -m 0755 -d /etc/apt/keyrings
|
||||
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
|
||||
sudo chmod a+r /etc/apt/keyrings/docker.asc
|
||||
|
||||
# Add the repository to Apt sources:
|
||||
echo \
|
||||
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
|
||||
$(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}") stable" | \
|
||||
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
|
||||
sudo apt-get update
|
||||
```
|
||||
|
||||
```bash
|
||||
sudo apt-get install containerd.io
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
# Cheat Sheet
|
||||
## **1. Images (Pull, List, Remove)**
|
||||
| Action | Docker | containerd (`ctr`) |
|
||||
|--------|--------|------------------|
|
||||
| Pull an image | `docker pull busybox` | `sudo ctr images pull docker.io/library/busybox:latest` |
|
||||
| List images | `docker images` | `sudo ctr images list` |
|
||||
| Remove an image | `docker rmi busybox` | `sudo ctr images rm docker.io/library/busybox:latest` |
|
||||
|
||||
---
|
||||
|
||||
## **2. Containers (Run, List, Stop, Remove)**
|
||||
| Action | Docker | containerd (`ctr`) |
|
||||
|--------|--------|------------------|
|
||||
| Run a container (interactive) | `docker run -it busybox sh` | `sudo ctr run --rm --tty docker.io/library/busybox:latest my-container sh` |
|
||||
| Run a container (detached) | `docker run -d busybox` | `sudo ctr run -d --tty docker.io/library/busybox:latest my-container sh` |
|
||||
| List running containers | `docker ps` | `sudo ctr tasks ls` |
|
||||
| Stop a running container | `docker stop my-container` | `sudo ctr tasks kill my-container` |
|
||||
| Remove a stopped container | `docker rm my-container` | `sudo ctr containers delete my-container` |
|
||||
|
||||
---
|
||||
|
||||
## **3. Container Execution (Attach, Exec, Logs)**
|
||||
| Action | Docker | containerd (`ctr`) |
|
||||
|--------|--------|------------------|
|
||||
| Attach to a running container | `docker attach my-container` | `sudo ctr tasks exec --tty my-container sh` |
|
||||
| Execute a command inside a container | `docker exec -it my-container sh` | `sudo ctr tasks exec --tty my-container sh` |
|
||||
| View container logs | `docker logs my-container` | **Not directly available in `ctr`** (use `crictl logs <container-id>` if using Kubernetes) |
|
||||
|
||||
---
|
||||
|
||||
## **4. Container Lifecycle Management**
|
||||
| Action | Docker | containerd (`ctr`) |
|
||||
|--------|--------|------------------|
|
||||
| Restart a container | `docker restart my-container` | `sudo ctr tasks kill -s SIGTERM my-container && sudo ctr run -d --tty docker.io/library/busybox:latest my-container sh` |
|
||||
| Pause a container | `docker pause my-container` | **Not directly supported** |
|
||||
| Unpause a container | `docker unpause my-container` | **Not directly supported** |
|
||||
|
||||
---
|
||||
|
||||
## **5. Volumes & Networking**
|
||||
Unlike Docker, `containerd` does **not** manage networking or volumes directly. You need to use **CNI (Container Network Interface)** for networking and manually mount host volumes when running a container.
|
||||
|
||||
For example, in Docker:
|
||||
```sh
|
||||
docker run -v /host/path:/container/path busybox
|
||||
```
|
||||
In `containerd`, you must specify the mount manually:
|
||||
```sh
|
||||
sudo ctr run --mount type=bind,src=/host/path,dst=/container/path,options=rbind:rw docker.io/library/busybox:latest my-container sh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## **6. Checking System Info**
|
||||
| Action | Docker | containerd (`ctr`) |
|
||||
|--------|--------|------------------|
|
||||
| Show system info | `docker info` | `sudo ctr version && sudo ctr plugins ls` |
|
||||
|
||||
---
|
||||
|
||||
## **7. Removing Everything (Clean Up)**
|
||||
| Action | Docker | containerd (`ctr`) |
|
||||
|--------|--------|------------------|
|
||||
| Remove all stopped containers | `docker container prune` | `sudo ctr containers list -q | xargs -r sudo ctr containers delete` |
|
||||
| Remove all images | `docker image prune -a` | `sudo ctr images list -q | xargs -r sudo ctr images rm` |
|
||||
|
||||
---
|
||||
|
||||
### **Summary**
|
||||
- `ctr` is more **low-level** than Docker, so it lacks built-in networking, volume management, and logging.
|
||||
- You must manually handle networking using **CNI**.
|
||||
- For Kubernetes workloads, you usually use `crictl` instead of `ctr`.
|
||||
- `containerd` focuses purely on running containers; it does not replace Docker Compose or Swarm.
|
||||
|
||||
@@ -0,0 +1,138 @@
|
||||
# Install kubeadm for Kubernetes version 1.32
|
||||
|
||||
## Prerquisites
|
||||
- A compatible Linux host
|
||||
- 2 GB or more of RAM per machine
|
||||
- 2 CPUs or more for control plane machines.
|
||||
- Full network connectivity between all machines in the cluster (public or private network is fine).
|
||||
- Unique hostname, MAC address, and product_uuid for every node.
|
||||
- Verify the `MAC address` and `product_uuid` are unique for every node
|
||||
You can get the `MAC address` of the network interfaces using the command `ip link` or `ifconfig -a`
|
||||
The `product_uuid `can be checked by using the command `sudo cat /sys/class/dmi/id/product_uuid`
|
||||
It is very likely that hardware devices will have unique addresses, although some virtual machines may have identical values.
|
||||
Kubernetes uses these values to uniquely identify the nodes in the cluster.
|
||||
If these values are not unique to each node, the installation process may fail.
|
||||
- Certain ports are open on your machines:
|
||||
|
||||
### Control plane
|
||||
|
||||
|Protocol|Direction|Port Range|Purpose|Used By|
|
||||
|---|---|---|---|---|
|
||||
|TCP|Inbound|6443|Kubernetes API server|All|
|
||||
|TCP|Inbound|2379-2380|etcd server client API|kube-apiserver, etcd|
|
||||
|TCP|Inbound|10250|Kubelet API|Self, Control plane|
|
||||
|TCP|Inbound|10259|kube-scheduler|Self|
|
||||
|TCP|Inbound|10257|kube-controller-manager|Self|
|
||||
Although etcd ports are included in control plane section,
|
||||
you can also host your own etcd cluster externally or on custom ports.
|
||||
|
||||
### Worker node(s)
|
||||
|Protocol|Direction|Port Range|Purpose| Used By |
|
||||
|---|---|---|---|---|
|
||||
|TCP|Inbound|10250|Kubelet API|Self, Control plane |
|
||||
|TCP|Inbound|10256|kube-proxy|Self, Load balancers |
|
||||
|TCP|Inbound|30000-32767|NodePort Services†|All|
|
||||
† Default port range for NodePort Services.
|
||||
|
||||
All default port numbers can be overridden.
|
||||
When custom ports are used those ports need to be open instead of defaults mentioned here.
|
||||
|
||||
One common example is API server port that is sometimes switched to 443.
|
||||
Alternatively, the default port is kept as is and API server is put behind a load balancer that listens on 443
|
||||
and routes the requests to API server on the default port.
|
||||
|
||||
### Check required ports
|
||||
These required ports need to be open in order for Kubernetes components to communicate with each other.
|
||||
You can use tools like netcat to check if a port is open. For example:
|
||||
```bash
|
||||
nc 127.0.0.1 6443 -v
|
||||
```
|
||||
The pod network plugin you use may also require certain ports to be open.
|
||||
Since this differs with each pod network plugin,
|
||||
please see the documentation for the plugins about what port(s) those need.
|
||||
|
||||
## Swap Configuration
|
||||
The default behavior of a kubelet is to fail to start if swap memory is detected on a node.
|
||||
This means that swap should either be disabled or tolerated by kubelet.
|
||||
|
||||
- To tolerate swap, add `failSwapOn:` false to kubelet configuration or as a command line argument.
|
||||
Note: even if `failSwapOn: false` is provided, workloads wouldn't have swap access by default.
|
||||
This can be changed by setting a `swapBehavior`, again in the kubelet configuration file.
|
||||
To use swap, set a `swapBehavior` other than the default `NoSwap ` setting.
|
||||
|
||||
- To disable swap, `sudo swapoff -a` can be used to disable swapping temporarily.
|
||||
To make this change persistent across reboots, make sure swap is disabled in config files like `/etc/fstab`, `systemd.swap`
|
||||
depending how it was configured on your system
|
||||
|
||||
## Install Container Runtime
|
||||
Use containerd
|
||||
|
||||
Make sure to remove anything docker/container related installations
|
||||
```bash
|
||||
for pkg in docker.io docker-doc docker-compose docker-compose-v2 podman-docker containerd runc; do sudo apt-get remove $pkg; done
|
||||
```
|
||||
|
||||
```bash
|
||||
# Add Docker's official GPG key:
|
||||
sudo apt-get update
|
||||
sudo apt-get install ca-certificates curl
|
||||
sudo install -m 0755 -d /etc/apt/keyrings
|
||||
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
|
||||
sudo chmod a+r /etc/apt/keyrings/docker.asc
|
||||
|
||||
# Add the repository to Apt sources:
|
||||
echo \
|
||||
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
|
||||
$(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}") stable" | \
|
||||
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
|
||||
sudo apt-get update
|
||||
```
|
||||
|
||||
```bash
|
||||
sudo apt-get install containerd.io
|
||||
```
|
||||
|
||||
## Install kubeadm, kubelet, and kubectl
|
||||
You will install these packages on all of your machines:
|
||||
|
||||
- `kubeadm`: the command to bootstrap the cluster.
|
||||
- `kubelet`: the component that runs on all of the machines in your cluster and does things like starting pods and containers.
|
||||
- `kubectl`: the command line util to talk to your cluster.
|
||||
|
||||
1. Update the apt package index and install packages needed to use the Kubernetes apt repository:
|
||||
|
||||
```bash
|
||||
sudo apt-get update
|
||||
# apt-transport-https may be a dummy package; if so, you can skip that package
|
||||
sudo apt-get install -y apt-transport-https ca-certificates curl gpg
|
||||
```
|
||||
|
||||
2. Download the public signing key for the Kubernetes package repositories.
|
||||
The same signing key is used for all repositories so you can disregard the version in the URL:
|
||||
|
||||
```bash
|
||||
# If the directory `/etc/apt/keyrings` does not exist, it should be created before the curl command, read the note below.
|
||||
# sudo mkdir -p -m 755 /etc/apt/keyrings
|
||||
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.32/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
|
||||
```
|
||||
3. Add the appropriate Kubernetes apt repository.
|
||||
Please note that this repository have packages only for Kubernetes 1.32;
|
||||
for other Kubernetes minor versions, you need to change the Kubernetes minor version in the URL to match your desired minor version
|
||||
(you should also check that you are reading the documentation for the version of Kubernetes that you plan to install).
|
||||
|
||||
```bash
|
||||
# This overwrites any existing configuration in /etc/apt/sources.list.d/kubernetes.list
|
||||
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.32/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list
|
||||
```
|
||||
4. Update the apt package index, install kubelet, kubeadm and kubectl, and pin their version:
|
||||
```bash
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y kubelet kubeadm kubectl
|
||||
sudo apt-mark hold kubelet kubeadm kubectl
|
||||
```
|
||||
|
||||
(Optional) Enable the kubelet service before running kubeadm:
|
||||
```bash
|
||||
sudo systemctl enable --now kubelet
|
||||
```
|
||||
The kubelet is now restarting every few seconds, as it waits in a crashloop for kubeadm to tell it what to do.
|
||||
@@ -0,0 +1,19 @@
|
||||
# minimal-postgres.yaml
|
||||
apiVersion: "acid.zalan.do/v1"
|
||||
kind: postgresql
|
||||
metadata:
|
||||
name: acid-minimal-cluster
|
||||
spec:
|
||||
teamId: "acid"
|
||||
volume:
|
||||
size: 1Gi
|
||||
numberOfInstances: 3 # One master and two replicas for HA
|
||||
postgresql:
|
||||
version: "15"
|
||||
resources:
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 100Mi
|
||||
limits:
|
||||
cpu: 500m
|
||||
memory: 500Mi
|
||||
Reference in New Issue
Block a user