kubernetes-the-hard-way/docs/09-bootstrapping-kubernetes...

286 lines
8.1 KiB
Markdown
Raw Normal View History

2017-08-29 00:19:25 +03:00
# Bootstrapping the Kubernetes Worker Nodes
2023-08-01 16:48:20 +03:00
In this lab you will bootstrap three Kubernetes worker nodes. The following components will be installed on each node: [containerd](https://github.com/containerd/containerd), [runc](https://github.com/opencontainers/runc), [container networking plugins](https://github.com/containernetworking/plugins), [crictl](https://github.com/kubernetes-sigs/cri-tools), [kube-proxy](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-proxy/), [kubectl](https://kubernetes.io/docs/reference/kubectl/), and [kubelet](https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/).
2017-08-29 00:19:25 +03:00
## Prerequisites
The commands in this lab must be run on each worker instance: `worker-0`, `worker-1`, and `worker-2`. Login to each worker instance using the `gcloud` command. Example:
```
gcloud compute ssh worker-0
```
### Running commands in parallel with tmux
2023-08-01 16:48:20 +03:00
[tmux](https://tmux.github.io/) can be used to run commands on multiple compute instances at the same time. See the [Running commands in parallel with tmux](./01-prerequisites.md#running-commands-in-parallel-with-tmux) section in the Prerequisites lab.
2017-08-29 00:19:25 +03:00
## Provisioning a Kubernetes Worker Node
2023-08-01 16:48:20 +03:00
Install the OS dependencies ([conntrack](https://conntrack-tools.netfilter.org/), [ipset](https://ipset.netfilter.org/), and [socat](socat)):
2017-08-29 00:19:25 +03:00
```
2023-08-01 16:48:20 +03:00
sudo apt-get update
sudo apt-get --yes install conntrack ipset socat
2017-08-29 00:19:25 +03:00
```
2017-10-02 06:37:09 +03:00
> The socat binary enables support for the `kubectl port-forward` command.
2017-08-29 00:19:25 +03:00
2019-09-14 21:41:56 +03:00
### Disable Swap
By default the kubelet will fail to start if [swap](https://help.ubuntu.com/community/SwapFaq) is enabled. It is [recommended](https://github.com/kubernetes/kubernetes/issues/7294) that swap be disabled to ensure Kubernetes can provide proper resource allocation and quality of service.
Verify if swap is enabled:
```
sudo swapon --show
```
2023-08-01 16:48:20 +03:00
If output is empty then swap is not enabled. If swap is enabled run the following command to disable swap immediately:
2019-09-14 21:41:56 +03:00
```
2023-08-01 16:48:20 +03:00
sudo swapoff --all
2019-09-14 21:41:56 +03:00
```
> To ensure swap remains off after reboot consult your Linux distro documentation.
2017-08-29 00:19:25 +03:00
### Download and Install Worker Binaries
```
2023-08-01 16:48:20 +03:00
curl --location \
2024-02-12 10:25:04 +03:00
--remote-name --time-cond containerd-1.7.13-linux-amd64.tar.gz \
https://github.com/containerd/containerd/releases/download/v1.7.13/containerd-1.7.13-linux-amd64.tar.gz \
2023-08-01 16:48:20 +03:00
--remote-name --time-cond containerd.service \
2024-02-12 10:25:04 +03:00
https://raw.githubusercontent.com/containerd/containerd/v1.7.13/containerd.service \
2023-08-01 16:48:20 +03:00
--output runc --time-cond runc \
2024-02-12 10:25:04 +03:00
https://github.com/opencontainers/runc/releases/download/v1.1.12/runc.amd64 \
--remote-name --time-cond cni-plugins-linux-amd64-v1.4.0.tgz \
https://github.com/containernetworking/plugins/releases/download/v1.4.0/cni-plugins-linux-amd64-v1.4.0.tgz \
--remote-name --time-cond crictl-v1.29.0-linux-amd64.tar.gz \
https://github.com/kubernetes-sigs/cri-tools/releases/download/v1.29.0/crictl-v1.29.0-linux-amd64.tar.gz \
2023-08-01 16:48:20 +03:00
--remote-name --time-cond kube-proxy \
2024-02-12 10:25:04 +03:00
https://dl.k8s.io/release/v1.29.1/bin/linux/amd64/kube-proxy \
2023-08-01 16:48:20 +03:00
--remote-name --time-cond kubectl \
2024-02-12 10:25:04 +03:00
https://dl.k8s.io/release/v1.29.1/bin/linux/amd64/kubectl \
2023-08-01 16:48:20 +03:00
--remote-name --time-cond kubelet \
2024-02-12 10:25:04 +03:00
https://dl.k8s.io/release/v1.29.1/bin/linux/amd64/kubelet
2017-08-29 00:19:25 +03:00
```
Create the installation directories:
```
2023-08-01 16:48:20 +03:00
sudo mkdir --parents \
2017-08-29 00:19:25 +03:00
/etc/cni/net.d \
/opt/cni/bin \
/var/lib/kube-proxy \
2023-08-01 16:48:20 +03:00
/var/lib/kubelet \
2017-08-29 00:19:25 +03:00
/var/lib/kubernetes \
/var/run/kubernetes
```
Install the worker binaries:
```
2023-08-01 16:48:20 +03:00
sudo tar --directory /usr/local/ --extract \
2024-02-12 10:25:04 +03:00
--file containerd-1.7.13-linux-amd64.tar.gz --gunzip --verbose
2017-08-29 00:19:25 +03:00
2023-08-01 16:48:20 +03:00
sudo mkdir --parents /usr/local/lib/systemd/system
2017-08-29 00:19:25 +03:00
2023-08-01 16:48:20 +03:00
sudo cp containerd.service /usr/local/lib/systemd/system/
2017-08-29 00:19:25 +03:00
2023-08-01 16:48:20 +03:00
sudo install --mode 0755 runc /usr/local/sbin/
2024-02-12 10:25:04 +03:00
tar --extract --file crictl-v1.29.0-linux-amd64.tar.gz --gunzip --verbose
2023-08-01 16:48:20 +03:00
sudo tar --directory /opt/cni/bin/ --extract \
2024-02-12 10:25:04 +03:00
--file cni-plugins-linux-amd64-v1.4.0.tgz --gunzip --verbose
2023-08-01 16:48:20 +03:00
sudo install --mode 0755 crictl kube-proxy kubectl kubelet /usr/local/bin/
2017-08-29 00:19:25 +03:00
```
2023-08-01 16:48:20 +03:00
### Configure CNI Networking
Retrieve the Pod CIDR range for the current compute instance:
2017-08-29 00:19:25 +03:00
```
2023-08-01 16:48:20 +03:00
POD_CIDR="$(curl --silent --header 'Metadata-Flavor: Google' \
http://metadata.google.internal/computeMetadata/v1/instance/attributes/pod-cidr)"
2017-08-29 00:19:25 +03:00
```
2023-08-01 16:48:20 +03:00
Create the CNI config file:
2017-08-29 00:19:25 +03:00
```
2023-08-01 16:48:20 +03:00
cat << EOF | sudo tee /etc/cni/net.d/10-containerd-net.conflist
2017-08-29 00:19:25 +03:00
{
2023-08-01 16:48:20 +03:00
"cniVersion": "1.0.0",
"name": "containerd-net",
"plugins": [
{
"type": "bridge",
"bridge": "cni0",
"isGateway": true,
"ipMasq": true,
"promiscMode": true,
"ipam": {
"type": "host-local",
"ranges": [
[{
"subnet": "${POD_CIDR}"
}]
],
"routes": [
{ "dst": "0.0.0.0/0" }
]
}
},
{
"type": "portmap",
"capabilities": {"portMappings": true},
"externalSetMarkChain": "KUBE-MARK-MASQ"
}
]
2017-08-29 00:19:25 +03:00
}
EOF
```
### Configure containerd
Create the `containerd` configuration file:
2017-08-29 00:19:25 +03:00
```
2023-08-01 16:48:20 +03:00
sudo mkdir --parents /etc/containerd
2017-08-29 00:19:25 +03:00
2023-08-01 16:48:20 +03:00
containerd config default | sudo tee /etc/containerd/config.toml
2017-08-29 00:19:25 +03:00
```
2023-08-01 16:48:20 +03:00
### Configure the Kubelet
2017-08-29 00:19:25 +03:00
```
2023-08-01 16:48:20 +03:00
sudo cp "${HOSTNAME}-key.pem" "${HOSTNAME}.pem" /var/lib/kubelet/
2023-08-01 16:48:20 +03:00
sudo cp "${HOSTNAME}.kubeconfig" /var/lib/kubelet/kubeconfig
2023-08-01 16:48:20 +03:00
sudo cp ca.pem /var/lib/kubernetes/
```
Create the `kubelet-config.yaml` configuration file:
```
cat <<EOF | sudo tee /var/lib/kubelet/kubelet-config.yaml
kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
authentication:
anonymous:
enabled: false
webhook:
enabled: true
x509:
clientCAFile: "/var/lib/kubernetes/ca.pem"
authorization:
mode: Webhook
clusterDomain: "cluster.local"
clusterDNS:
- "10.32.0.10"
podCIDR: "${POD_CIDR}"
resolvConf: "/run/systemd/resolve/resolv.conf"
runtimeRequestTimeout: "15m"
tlsCertFile: "/var/lib/kubelet/${HOSTNAME}.pem"
tlsPrivateKeyFile: "/var/lib/kubelet/${HOSTNAME}-key.pem"
EOF
2017-08-29 00:19:25 +03:00
```
2023-08-01 16:48:20 +03:00
> The `resolvConf` configuration is used to avoid loops when using CoreDNS for service discovery on systems running `systemd-resolved`.
2017-08-29 00:19:25 +03:00
Create the `kubelet.service` systemd unit file:
```
2023-08-01 16:48:20 +03:00
cat <<EOF | sudo tee /usr/local/lib/systemd/system/kubelet.service
2017-08-29 00:19:25 +03:00
[Unit]
Description=Kubernetes Kubelet
2017-12-18 18:07:54 +03:00
Documentation=https://github.com/kubernetes/kubernetes
After=containerd.service
Requires=containerd.service
2017-08-29 00:19:25 +03:00
[Service]
ExecStart=/usr/local/bin/kubelet \\
2023-08-01 16:48:20 +03:00
--config /var/lib/kubelet/kubelet-config.yaml \\
--container-runtime-endpoint unix:///var/run/containerd/containerd.sock \\
--kubeconfig /var/lib/kubelet/kubeconfig \\
--register-node \\
--v 2
2017-08-29 00:19:25 +03:00
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
EOF
```
### Configure the Kubernetes Proxy
```
2023-08-01 16:48:20 +03:00
sudo cp kube-proxy.kubeconfig /var/lib/kube-proxy/kubeconfig
2017-08-29 00:19:25 +03:00
```
Create the `kube-proxy-config.yaml` configuration file:
```
cat <<EOF | sudo tee /var/lib/kube-proxy/kube-proxy-config.yaml
kind: KubeProxyConfiguration
apiVersion: kubeproxy.config.k8s.io/v1alpha1
clientConnection:
kubeconfig: "/var/lib/kube-proxy/kubeconfig"
mode: "iptables"
clusterCIDR: "10.200.0.0/16"
EOF
```
2017-08-29 00:19:25 +03:00
Create the `kube-proxy.service` systemd unit file:
```
2023-08-01 16:48:20 +03:00
cat <<EOF | sudo tee /usr/local/lib/systemd/system/kube-proxy.service
2017-08-29 00:19:25 +03:00
[Unit]
Description=Kubernetes Kube Proxy
2017-12-18 18:07:54 +03:00
Documentation=https://github.com/kubernetes/kubernetes
2017-08-29 00:19:25 +03:00
[Service]
ExecStart=/usr/local/bin/kube-proxy \\
2023-08-01 16:48:20 +03:00
--config /var/lib/kube-proxy/kube-proxy-config.yaml
2017-08-29 00:19:25 +03:00
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
EOF
```
### Start the Worker Services
```
2023-08-01 16:48:20 +03:00
sudo systemctl enable --now containerd kubelet kube-proxy
2017-08-29 00:19:25 +03:00
```
> Remember to run the above commands on each worker node: `worker-0`, `worker-1`, and `worker-2`.
## Verification
> The compute instances created in this tutorial will not have permission to complete this section. Run the following commands from the same machine used to create the compute instances.
2017-08-29 00:19:25 +03:00
List the registered Kubernetes nodes:
```
gcloud compute ssh controller-0 \
2023-08-01 16:48:20 +03:00
--command 'kubectl get nodes --kubeconfig admin.kubeconfig'
2017-08-29 00:19:25 +03:00
```
> output
```
NAME STATUS ROLES AGE VERSION
2024-02-12 10:25:04 +03:00
worker-0 Ready <none> 37s v1.29.1
worker-1 Ready <none> 37s v1.29.1
worker-2 Ready <none> 37s v1.29.1
2017-08-29 00:19:25 +03:00
```
2023-08-01 16:48:20 +03:00
Next: [Configuring kubectl for Remote Access](./10-configuring-kubectl.md)