mirror of
https://github.com/kelseyhightower/kubernetes-the-hard-way.git
synced 2025-12-15 01:08:58 +03:00
Upgrade/1.24 (#291)
* Set up Vagrantfile - Use Ubuntu 22.04 - Set required kernel parameters and tunables - Optimise file for DRY by use of local functions - No longer install Docker * Update prerequisites * Update compute resources * Update client-tools * Update cert authority * Update kube config files * Update sata encryption keys * Update etcd * Cert enhancements - Use dig for host IPs - Create front-proxy keys * Update prereqs with lab defaults * Minor update * Dynamic kubelet reconfig removed in 1.24 * Update failed provisioning * Update cert sujects. Use vars for IP addresses * Use vars for IP addresses * USe vars for IPs. Update unit file * Unit updates for 1.24. Use vars for IPs * 1.24 changes - Update unit files - Use vars for IPs - Install containerd * Use vars for IPs. Update outputs * Remove CNI plugins - done earlier * Update API versions * Adjust VM RAM * Update coredns version and api versions * Update git ignore and attributes * Note about deprecation warning * Fix kubeconfig name * Formatting changes + pin nginx version * Update kubetest * Update README * Discuss why only 2 masters * Note on changing service cidr range vs coredns * Add RAM column to VM table * Best practice - secure PKI * Secure kubeconfig * Add prev link * Adding `Prev` links * Squashed commit of the following: commit 8fbd36069cbf7365f627e5ebf5a04e37cde085d9 Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com> Date: Thu Aug 25 20:06:10 2022 +0100 Update dns-addon test commit 5528e873ecbe3265155da48d24c24d696635af52 Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com> Date: Thu Aug 25 20:00:48 2022 +0100 Fix get nodes commit 0d88ab0d1c4b6a7ae05bc2552366460f741bb763 Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com> Date: Thu Aug 25 20:00:19 2022 +0100 Fix env var name commit e564db03ff9c4c9ef536bcc5cd999fa1e6a3de15 Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com> Date: Thu Aug 25 19:42:52 2022 +0100 Update e2e-tests commit 247a59f2c5b84e34972f396cf87a34bcbeb2d2ef Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com> Date: Thu Aug 25 19:39:54 2022 +0100 Updated e2e-tests commit 60b33d025bb252570f41c13f90955ec8d59141a7 Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com> Date: Thu Aug 25 19:38:02 2022 +0100 bashify commands in ``` commit 2814949d6dd569c59ea7ec61135784d51ad4de1f Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com> Date: Thu Aug 25 19:35:32 2022 +0100 Note deprecation warning when deploying weave commit af0264e13e5f0e277f8f31e5115a813680aadd74 Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com> Date: Thu Aug 25 19:33:55 2022 +0100 Nodes are ready at end of step 11 commit 050502386d36a8593ed7348e902cdff9ad9c64b2 Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com> Date: Thu Aug 25 19:30:00 2022 +0100 Minor change CNI commit 04bdc1483e9696ed018ac26b6480237ee1dcf1d1 Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com> Date: Thu Aug 25 19:21:22 2022 +0100 Explain data at rest is in etcd commit 243154b9866f5a7a1a49037f97e38c6bf7ffbcb7 Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com> Date: Thu Aug 25 19:18:49 2022 +0100 Explanation of api cluster ip commit dd168ac2e128cbd405248115d8724498fa18fa67 Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com> Date: Thu Aug 25 19:14:42 2022 +0100 Include vagrant password commit d51c65a77ac192e2468d92f0067958c69057a2e0 Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com> Date: Thu Aug 25 19:12:34 2022 +0100 Update tmux message commit 10f41737100ab410adb6b20712ee32cd80618e3d Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com> Date: Thu Aug 25 19:09:23 2022 +0100 Insert step to configure CNI on both workers Optionally with tmux commit 8fd873f1492f6ea1c846b3309f57740e8501adee Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com> Date: Thu Aug 25 18:42:27 2022 +0100 Shuffle up to make room for common cni install commit d650443b069a7543cbb4cf449818a81d84932007 Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com> Date: Thu Aug 25 07:34:59 2022 +0100 Added warning output to componentstatuses commit 7bfef8f16bd1a126dcf3e5f43a02d79517d64c74 Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com> Date: Thu Aug 25 07:34:38 2022 +0100 Rearrange text commit b16b92bc6513cf355a41afa22ddfe2696142c28b Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com> Date: Thu Aug 25 07:34:18 2022 +0100 Minor wording change DNS arress is conventionally .10 commit 96c9d25663ce3d721e670262bb6858e9a7183873 Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com> Date: Thu Aug 25 07:32:24 2022 +0100 Use shell vars for etcd addresses commit c9e223fba5324a1c65d6f583cf9e739b8459df5d Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com> Date: Thu Aug 25 07:31:58 2022 +0100 Update on network defaults commit 1cf98649df9410b8a7d14c68bcb17c24aa6a210a Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com> Date: Thu Aug 25 07:05:38 2022 +0100 Get and install correct CNI components commit 311905fba72f4a48cde4a73c589daea9b76042b7 Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com> Date: Thu Aug 25 06:18:55 2022 +0100 Update Approve CSR commit 4c39c84c172fde8ab2aafc4ea38b050eb7f3019b Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com> Date: Wed Aug 24 20:34:53 2022 +0100 Moving certs out of service kuebeconfigs * Squashed commit of the following: commit 252cc335739e3c8007ab86c951222aba954d80f7 Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com> Date: Sun Aug 28 20:29:23 2022 +0100 Update external links commit 8091d1a13bc5a29654db2b8fecd55b8180bf8cab Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com> Date: Sun Aug 28 20:28:14 2022 +0100 Mac M1 note commit 8b7e6065ffb74532b6ad7570a8c978addcc7fb66 Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com> Date: Sun Aug 28 20:03:11 2022 +0100 Tweak order of commands e2e tests commit 857d039dd1dff28e92d392ad6c5e40814a9eb054 Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com> Date: Sun Aug 28 20:02:51 2022 +0100 Fixing kubecomfig checks commit 26f42049bebd2d539406e6e16c51bb06441702f1 Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com> Date: Sun Aug 28 15:51:13 2022 +0100 Updated cert_verify commit 0df54e4c3499e6d79b836e1dfcf74eb9fdf196b1 Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com> Date: Sun Aug 28 09:09:14 2022 +0100 Rewite cert_verify Round 1 certs and kubeconfigs * Update README - Insert CNI lab - Correct CNI versions * Automate hostfile network settings Determine from interface address passed in. * Update 01-prerequisites.md * Update 01-prerequisites.md Correct the default vm ip range * Review updates. Issue 1 * Review updates. Issue 2 * Review updates. Issue 3 In actual fact, the base script is cert_verfiy.sh so the error is in the link created by the provisioner. You'll see that the later labs all refer to it with underscore. * Review updates. Issue 5 * Review updates. Issue 6 * Review updates. Issue 7 I whip through the scripts so fast, that even if I had copied it twice to my quick script, I didn't notice it say that the resource exists and is unchanged! * These certs already copied in step 4 * Formatting and command grouping * Review updates. Step 11 cert_verify Needs to be done after kublet starts as it is looking for the auto-issued cert * Group coomand batches * Remove duplicate clusterrolebinding * Extraction of scripts from md using tool This uses markdown comments and ```bash fence to determine what to extract and for which hosts Fixed shell var bug in step 11 * Fixed typos * Be specific that we're doing shutdown, not suspend * Minor edits for clarity * remove the extra \ * Rename step 9 to CRI, as that's what it actually is * Disambiguate CRI vs CNI * small fixes Co-authored-by: Tej Singh Rana <58101587+Tej-Singh-Rana@users.noreply.github.com>
This commit is contained in:
Alistair Mackay
303
docs/10-bootstrapping-kubernetes-workers.md
Normal file
303
docs/10-bootstrapping-kubernetes-workers.md
Normal file
@@ -0,0 +1,303 @@
|
||||
# Bootstrapping the Kubernetes Worker Nodes
|
||||
|
||||
In this lab you will bootstrap 2 Kubernetes worker nodes. We already installed `containerd` and its dependencies on these nodes in the previous lab.
|
||||
|
||||
We will now install the kubernetes components
|
||||
- [kubelet](https://kubernetes.io/docs/admin/kubelet)
|
||||
- [kube-proxy](https://kubernetes.io/docs/concepts/cluster-administration/proxies).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
The Certificates and Configuration are created on `master-1` node and then copied over to workers using `scp`.
|
||||
Once this is done, the commands are to be run on first worker instance: `worker-1`. Login to first worker instance using SSH Terminal.
|
||||
|
||||
### Provisioning Kubelet Client Certificates
|
||||
|
||||
Kubernetes uses a [special-purpose authorization mode](https://kubernetes.io/docs/admin/authorization/node/) called Node Authorizer, that specifically authorizes API requests made by [Kubelets](https://kubernetes.io/docs/concepts/overview/components/#kubelet). In order to be authorized by the Node Authorizer, Kubelets must use a credential that identifies them as being in the `system:nodes` group, with a username of `system:node:<nodeName>`. In this section you will create a certificate for each Kubernetes worker node that meets the Node Authorizer requirements.
|
||||
|
||||
Generate a certificate and private key for one worker node:
|
||||
|
||||
On `master-1`:
|
||||
|
||||
[//]: # (host:master-1)
|
||||
|
||||
```bash
|
||||
WORKER_1=$(dig +short worker-1)
|
||||
```
|
||||
|
||||
```bash
|
||||
cat > openssl-worker-1.cnf <<EOF
|
||||
[req]
|
||||
req_extensions = v3_req
|
||||
distinguished_name = req_distinguished_name
|
||||
[req_distinguished_name]
|
||||
[ v3_req ]
|
||||
basicConstraints = CA:FALSE
|
||||
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
|
||||
subjectAltName = @alt_names
|
||||
[alt_names]
|
||||
DNS.1 = worker-1
|
||||
IP.1 = ${WORKER_1}
|
||||
EOF
|
||||
|
||||
openssl genrsa -out worker-1.key 2048
|
||||
openssl req -new -key worker-1.key -subj "/CN=system:node:worker-1/O=system:nodes" -out worker-1.csr -config openssl-worker-1.cnf
|
||||
openssl x509 -req -in worker-1.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out worker-1.crt -extensions v3_req -extfile openssl-worker-1.cnf -days 1000
|
||||
```
|
||||
|
||||
Results:
|
||||
|
||||
```
|
||||
worker-1.key
|
||||
worker-1.crt
|
||||
```
|
||||
|
||||
### The kubelet Kubernetes Configuration File
|
||||
|
||||
When generating kubeconfig files for Kubelets the client certificate matching the Kubelet's node name must be used. This will ensure Kubelets are properly authorized by the Kubernetes [Node Authorizer](https://kubernetes.io/docs/admin/authorization/node/).
|
||||
|
||||
Get the kub-api server load-balancer IP.
|
||||
|
||||
```bash
|
||||
LOADBALANCER=$(dig +short loadbalancer)
|
||||
```
|
||||
|
||||
Generate a kubeconfig file for the first worker node.
|
||||
|
||||
On `master-1`:
|
||||
```bash
|
||||
{
|
||||
kubectl config set-cluster kubernetes-the-hard-way \
|
||||
--certificate-authority=/var/lib/kubernetes/pki/ca.crt \
|
||||
--server=https://${LOADBALANCER}:6443 \
|
||||
--kubeconfig=worker-1.kubeconfig
|
||||
|
||||
kubectl config set-credentials system:node:worker-1 \
|
||||
--client-certificate=/var/lib/kubernetes/pki/worker-1.crt \
|
||||
--client-key=/var/lib/kubernetes/pki/worker-1.key \
|
||||
--kubeconfig=worker-1.kubeconfig
|
||||
|
||||
kubectl config set-context default \
|
||||
--cluster=kubernetes-the-hard-way \
|
||||
--user=system:node:worker-1 \
|
||||
--kubeconfig=worker-1.kubeconfig
|
||||
|
||||
kubectl config use-context default --kubeconfig=worker-1.kubeconfig
|
||||
}
|
||||
```
|
||||
|
||||
Results:
|
||||
|
||||
```
|
||||
worker-1.kubeconfig
|
||||
```
|
||||
|
||||
### Copy certificates, private keys and kubeconfig files to the worker node:
|
||||
On `master-1`:
|
||||
|
||||
```bash
|
||||
scp ca.crt worker-1.crt worker-1.key worker-1.kubeconfig worker-1:~/
|
||||
```
|
||||
|
||||
|
||||
### Download and Install Worker Binaries
|
||||
|
||||
All the following commands from here until the [verification](#verification) step must be run on `worker-1`
|
||||
|
||||
[//]: # (host:worker-1)
|
||||
|
||||
|
||||
```bash
|
||||
wget -q --show-progress --https-only --timestamping \
|
||||
https://storage.googleapis.com/kubernetes-release/release/v1.24.3/bin/linux/amd64/kubectl \
|
||||
https://storage.googleapis.com/kubernetes-release/release/v1.24.3/bin/linux/amd64/kube-proxy \
|
||||
https://storage.googleapis.com/kubernetes-release/release/v1.24.3/bin/linux/amd64/kubelet
|
||||
```
|
||||
|
||||
Reference: https://kubernetes.io/releases/download/#binaries
|
||||
|
||||
Create the installation directories:
|
||||
|
||||
```bash
|
||||
sudo mkdir -p \
|
||||
/var/lib/kubelet \
|
||||
/var/lib/kube-proxy \
|
||||
/var/lib/kubernetes/pki \
|
||||
/var/run/kubernetes
|
||||
```
|
||||
|
||||
Install the worker binaries:
|
||||
|
||||
```bash
|
||||
{
|
||||
chmod +x kubectl kube-proxy kubelet
|
||||
sudo mv kubectl kube-proxy kubelet /usr/local/bin/
|
||||
}
|
||||
```
|
||||
|
||||
### Configure the Kubelet
|
||||
On worker-1:
|
||||
|
||||
Copy keys and config to correct directories and secure
|
||||
|
||||
```bash
|
||||
{
|
||||
sudo mv ${HOSTNAME}.key ${HOSTNAME}.crt /var/lib/kubernetes/pki/
|
||||
sudo mv ${HOSTNAME}.kubeconfig /var/lib/kubelet/kubelet.kubeconfig
|
||||
sudo mv ca.crt /var/lib/kubernetes/pki/
|
||||
sudo mv kube-proxy.crt kube-proxy.key /var/lib/kubernetes/pki/
|
||||
sudo chown root:root /var/lib/kubernetes/pki/*
|
||||
sudo chmod 600 /var/lib/kubernetes/pki/*
|
||||
sudo chown root:root /var/lib/kubelet/*
|
||||
sudo chmod 600 /var/lib/kubelet/*
|
||||
}
|
||||
```
|
||||
|
||||
CIDR ranges used *within* the cluster
|
||||
|
||||
```bash
|
||||
POD_CIDR=10.244.0.0/16
|
||||
SERVICE_CIDR=10.96.0.0/16
|
||||
```
|
||||
|
||||
Compute cluster DNS addess, which is conventionally .10 in the service CIDR range
|
||||
|
||||
```bash
|
||||
CLUSTER_DNS=$(echo $SERVICE_CIDR | awk 'BEGIN {FS="."} ; { printf("%s.%s.%s.10", $1, $2, $3) }')
|
||||
```
|
||||
|
||||
Create the `kubelet-config.yaml` configuration file:
|
||||
|
||||
```bash
|
||||
cat <<EOF | sudo tee /var/lib/kubelet/kubelet-config.yaml
|
||||
kind: KubeletConfiguration
|
||||
apiVersion: kubelet.config.k8s.io/v1beta1
|
||||
authentication:
|
||||
anonymous:
|
||||
enabled: false
|
||||
webhook:
|
||||
enabled: true
|
||||
x509:
|
||||
clientCAFile: /var/lib/kubernetes/pki/ca.crt
|
||||
authorization:
|
||||
mode: Webhook
|
||||
clusterDomain: cluster.local
|
||||
clusterDNS:
|
||||
- ${CLUSTER_DNS}
|
||||
resolvConf: /run/systemd/resolve/resolv.conf
|
||||
runtimeRequestTimeout: "15m"
|
||||
tlsCertFile: /var/lib/kubernetes/pki/${HOSTNAME}.crt
|
||||
tlsPrivateKeyFile: /var/lib/kubernetes/pki/${HOSTNAME}.key
|
||||
registerNode: true
|
||||
EOF
|
||||
```
|
||||
|
||||
> The `resolvConf` configuration is used to avoid loops when using CoreDNS for service discovery on systems running `systemd-resolved`.
|
||||
|
||||
Create the `kubelet.service` systemd unit file:
|
||||
|
||||
```bash
|
||||
cat <<EOF | sudo tee /etc/systemd/system/kubelet.service
|
||||
[Unit]
|
||||
Description=Kubernetes Kubelet
|
||||
Documentation=https://github.com/kubernetes/kubernetes
|
||||
After=containerd.service
|
||||
Requires=containerd.service
|
||||
|
||||
[Service]
|
||||
ExecStart=/usr/local/bin/kubelet \\
|
||||
--config=/var/lib/kubelet/kubelet-config.yaml \\
|
||||
--container-runtime-endpoint=unix:///var/run/containerd/containerd.sock \\
|
||||
--kubeconfig=/var/lib/kubelet/kubelet.kubeconfig \\
|
||||
--v=2
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
EOF
|
||||
```
|
||||
|
||||
### Configure the Kubernetes Proxy
|
||||
On worker-1:
|
||||
|
||||
```bash
|
||||
sudo mv kube-proxy.kubeconfig /var/lib/kube-proxy/
|
||||
```
|
||||
|
||||
Create the `kube-proxy-config.yaml` configuration file:
|
||||
|
||||
```bash
|
||||
cat <<EOF | sudo tee /var/lib/kube-proxy/kube-proxy-config.yaml
|
||||
kind: KubeProxyConfiguration
|
||||
apiVersion: kubeproxy.config.k8s.io/v1alpha1
|
||||
clientConnection:
|
||||
kubeconfig: "/var/lib/kube-proxy/kube-proxy.kubeconfig"
|
||||
mode: "iptables"
|
||||
clusterCIDR: ${POD_CIDR}
|
||||
EOF
|
||||
```
|
||||
|
||||
Create the `kube-proxy.service` systemd unit file:
|
||||
|
||||
```bash
|
||||
cat <<EOF | sudo tee /etc/systemd/system/kube-proxy.service
|
||||
[Unit]
|
||||
Description=Kubernetes Kube Proxy
|
||||
Documentation=https://github.com/kubernetes/kubernetes
|
||||
|
||||
[Service]
|
||||
ExecStart=/usr/local/bin/kube-proxy \\
|
||||
--config=/var/lib/kube-proxy/kube-proxy-config.yaml
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
EOF
|
||||
```
|
||||
|
||||
## Optional - Check Certificates and kubeconfigs
|
||||
|
||||
At `worker-1` node, run the following, selecting option 4
|
||||
|
||||
```bash
|
||||
./cert_verify.sh
|
||||
```
|
||||
|
||||
|
||||
### Start the Worker Services
|
||||
On worker-1:
|
||||
```bash
|
||||
{
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable kubelet kube-proxy
|
||||
sudo systemctl start kubelet kube-proxy
|
||||
}
|
||||
```
|
||||
|
||||
> Remember to run the above commands on worker node: `worker-1`
|
||||
|
||||
## Verification
|
||||
|
||||
[//]: # (host:master-1)
|
||||
|
||||
Now return to the `master-1` node.
|
||||
|
||||
List the registered Kubernetes nodes from the master node:
|
||||
|
||||
```bash
|
||||
kubectl get nodes --kubeconfig admin.kubeconfig
|
||||
```
|
||||
|
||||
> output
|
||||
|
||||
```
|
||||
NAME STATUS ROLES AGE VERSION
|
||||
worker-1 NotReady <none> 93s v1.24.3
|
||||
```
|
||||
|
||||
The node is not ready as we have not yet installed pod networking. This comes later.
|
||||
|
||||
Prev: [Installing CRI on the Kubernetes Worker Nodes](09-install-cri-workers.md)<br>
|
||||
Next: [TLS Bootstrapping Kubernetes Workers](11-tls-bootstrapping-kubernetes-workers.md)
|
||||
Reference in New Issue
Block a user