diff --git a/docs/08-bootstrapping-kubernetes-controllers.md b/docs/08-bootstrapping-kubernetes-controllers.md index 863a49d..a9ae867 100644 --- a/docs/08-bootstrapping-kubernetes-controllers.md +++ b/docs/08-bootstrapping-kubernetes-controllers.md @@ -1,13 +1,13 @@ # Bootstrapping the Kubernetes Control Plane -In this lab you will bootstrap the Kubernetes control plane across three compute instances and configure it for high availability. You will also create an external load balancer that exposes the Kubernetes API Servers to remote clients. The following components will be installed on each node: Kubernetes API Server, Scheduler, and Controller Manager. +In this lab you will bootstrap the Kubernetes control plane across three VM instances and configure it for high availability. You will also create a load balancer that exposes the Kubernetes API Servers to remote clients. The following components will be installed on each node: Kubernetes API Server, Scheduler, and Controller Manager. ## Prerequisites -The commands in this lab must be run on each controller instance: `controller-0`, `controller-1`, and `controller-2`. Login to each controller instance using the `gcloud` command. Example: +The commands in this lab must be run on each controller instance: `controller-0`, `controller-1`, and `controller-2`. Login to each controller instance using the `ssh` command. Example: ```bash -gcloud compute ssh controller-0 +ssh controller-0 ``` ### Running commands in parallel with tmux @@ -37,31 +37,28 @@ wget -q --show-progress --https-only --timestamping \ Install the Kubernetes binaries: ```bash -{ - chmod +x kube-apiserver kube-controller-manager kube-scheduler kubectl - sudo mv kube-apiserver kube-controller-manager kube-scheduler kubectl /usr/local/bin/ -} +chmod +x kube-apiserver kube-controller-manager kube-scheduler kubectl +sudo mv kube-apiserver kube-controller-manager kube-scheduler kubectl /usr/local/bin/ ``` ### Configure the Kubernetes API Server ```bash -{ - sudo mkdir -p /var/lib/kubernetes/ +sudo mkdir -p /var/lib/kubernetes/ - sudo mv ca.pem ca-key.pem kubernetes-key.pem kubernetes.pem \ - service-account-key.pem service-account.pem \ - encryption-config.yaml /var/lib/kubernetes/ -} +sudo mv ca.pem ca-key.pem kubernetes-key.pem kubernetes.pem \ + service-account-key.pem service-account.pem \ + encryption-config.yaml /var/lib/kubernetes/ ``` -The instance internal IP address will be used to advertise the API Server to members of the cluster. Retrieve the internal IP address for the current compute instance: +The instance internal IP address will be used to advertise the API Server to members of the cluster. Define the INTERNAL_IP (replace MY_NODE_INTERNAL_IP by the value): ```bash -INTERNAL_IP=$(curl -s -H "Metadata-Flavor: Google" \ - http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/ip) +INTERNAL_IP=MY_NODE_INTERNAL_IP ``` +> Example for controller-0 : 192.168.8.10 + Create the `kube-apiserver.service` systemd unit file: ```bash @@ -86,7 +83,7 @@ ExecStart=/usr/local/bin/kube-apiserver \\ --etcd-cafile=/var/lib/kubernetes/ca.pem \\ --etcd-certfile=/var/lib/kubernetes/kubernetes.pem \\ --etcd-keyfile=/var/lib/kubernetes/kubernetes-key.pem \\ - --etcd-servers=https://10.240.0.10:2379,https://10.240.0.11:2379,https://10.240.0.12:2379 \\ + --etcd-servers=https://192.168.8.10:2379,https://192.168.8.11:2379,https://192.168.8.12:2379 \\ --event-ttl=1h \\ --encryption-provider-config=/var/lib/kubernetes/encryption-config.yaml \\ --kubelet-certificate-authority=/var/lib/kubernetes/ca.pem \\ @@ -190,59 +187,13 @@ EOF ### Start the Controller Services ```bash -{ - sudo systemctl daemon-reload - sudo systemctl enable kube-apiserver kube-controller-manager kube-scheduler - sudo systemctl start kube-apiserver kube-controller-manager kube-scheduler -} +sudo systemctl daemon-reload +sudo systemctl enable kube-apiserver kube-controller-manager kube-scheduler +sudo systemctl start kube-apiserver kube-controller-manager kube-scheduler ``` > Allow up to 10 seconds for the Kubernetes API Server to fully initialize. -### Enable HTTP Health Checks - -A [Google Network Load Balancer](https://cloud.google.com/compute/docs/load-balancing/network) will be used to distribute traffic across the three API servers and allow each API server to terminate TLS connections and validate client certificates. The network load balancer only supports HTTP health checks which means the HTTPS endpoint exposed by the API server cannot be used. As a workaround the nginx webserver can be used to proxy HTTP health checks. In this section nginx will be installed and configured to accept HTTP health checks on port `80` and proxy the connections to the API server on `https://127.0.0.1:6443/healthz`. - -> The `/healthz` API server endpoint does not require authentication by default. - -Install a basic web server to handle HTTP health checks: - -```bash -sudo apt-get update -sudo apt-get install -y nginx -``` - -```bash -cat > kubernetes.default.svc.cluster.local < The compute instances created in this tutorial will not have permission to complete this section. **Run the following commands from the same machine used to create the compute instances**. +### Provision an Nginx Load Balancer -### Provision a Network Load Balancer - -Create the external load balancer network resources: +Install the Nginx Load Balancer: ```bash -{ - KUBERNETES_PUBLIC_ADDRESS=$(gcloud compute addresses describe kubernetes-the-hard-way \ - --region $(gcloud config get-value compute/region) \ - --format 'value(address)') - - gcloud compute http-health-checks create kubernetes \ - --description "Kubernetes Health Check" \ - --host "kubernetes.default.svc.cluster.local" \ - --request-path "/healthz" - - gcloud compute firewall-rules create kubernetes-the-hard-way-allow-health-check \ - --network kubernetes-the-hard-way \ - --source-ranges 209.85.152.0/22,209.85.204.0/22,35.191.0.0/16 \ - --allow tcp - - gcloud compute target-pools create kubernetes-target-pool \ - --http-health-check kubernetes - - gcloud compute target-pools add-instances kubernetes-target-pool \ - --instances controller-0,controller-1,controller-2 - - gcloud compute forwarding-rules create kubernetes-forwarding-rule \ - --address ${KUBERNETES_PUBLIC_ADDRESS} \ - --ports 6443 \ - --region $(gcloud config get-value compute/region) \ - --target-pool kubernetes-target-pool -} +sudo apt-get update +sudo apt-get install -y nginx ``` -### LB Verification - -> The compute instances created in this tutorial will not have permission to complete this section. **Run the following commands from the same machine used to create the compute instances**. - -Retrieve the `kubernetes-the-hard-way` static IP address: +Create the Nginx load balancer network configuration: ```bash -KUBERNETES_PUBLIC_ADDRESS=$(gcloud compute addresses describe kubernetes-the-hard-way \ - --region $(gcloud config get-value compute/region) \ - --format 'value(address)') +cat <> /etc/nginx/nginx.conf +stream { + upstream controller_backend { + server 192.168.8.10:6443; + server 192.168.8.11:6443; + server 192.168.8.12:6443; + } + server { + listen 6443; + proxy_pass controller_backend; + health_check; + } +} +EOF +``` + +Restart the service: + +```bash +sudo systemctl restart nginx +``` + +Enable the service: + +```bash +sudo systemctl enable nginx +``` + +### Load Balancer Verification + +Define the static public IP address (replace MY_PUBLIC_IP_ADDRESS with your public IP address on the `gateway-01` VM): + +```bash +KUBERNETES_PUBLIC_ADDRESS=MY_PUBLIC_IP_ADDRESS ``` Make a HTTP request for the Kubernetes version info: