kubernetes-the-hard-way/docs/05-kubernetes-controller.md

310 lines
7.9 KiB
Markdown
Raw Normal View History

2016-07-07 18:25:27 +03:00
# Bootstrapping an H/A Kubernetes Control Plane
2016-07-07 17:15:59 +03:00
2016-07-07 18:25:27 +03:00
In this lab you will bootstrap a 3 node Kubernetes controller cluster. The following virtual machines will be used:
2016-09-11 06:19:45 +03:00
* controller0
* controller1
* controller2
2016-07-07 17:15:59 +03:00
2016-07-08 00:37:13 +03:00
In this lab you will also create a frontend load balancer with a public IP address for remote access to the API servers and H/A.
2016-07-08 00:36:48 +03:00
2016-07-07 18:30:15 +03:00
## Why
The Kubernetes components that make up the control plane include the following components:
2017-03-25 21:41:26 +03:00
* API Server
* Scheduler
* Controller Manager
2016-07-07 18:30:15 +03:00
2017-03-25 21:41:26 +03:00
Each component is being run on the same machine for the following reasons:
2016-07-07 18:30:15 +03:00
* The Scheduler and Controller Manager are tightly coupled with the API Server
* Only one Scheduler and Controller Manager can be active at a given time, but it's ok to run multiple at the same time. Each component will elect a leader via the API Server.
* Running multiple copies of each component is required for H/A
* Running each component next to the API Server eases configuration.
2016-07-07 18:25:27 +03:00
2017-03-24 05:48:14 +03:00
## Provision the Kubernetes Controller Cluster
2016-07-09 19:28:12 +03:00
2017-03-24 05:48:14 +03:00
Run the following commands on `controller0`, `controller1`, `controller2`:
2016-07-09 19:28:12 +03:00
> Login to each machine using the gcloud compute ssh command
---
2017-03-24 05:48:14 +03:00
Copy the bootstrap token into place:
2016-07-09 19:28:12 +03:00
2017-03-24 09:08:54 +03:00
```
sudo mkdir -p /var/lib/kubernetes/
```
2017-03-24 05:48:14 +03:00
```
sudo mv token.csv /var/lib/kubernetes/
```
2016-09-11 17:31:04 +03:00
2017-03-24 05:48:14 +03:00
### TLS Certificates
2016-09-11 17:31:04 +03:00
2017-03-24 05:48:14 +03:00
The TLS certificates created in the [Setting up a CA and TLS Cert Generation](02-certificate-authority.md) lab will be used to secure communication between the Kubernetes API server and Kubernetes clients such as `kubectl` and the `kubelet` agent. The TLS certificates will also be used to authenticate the Kubernetes API server to etcd via TLS client auth.
2016-09-11 17:31:04 +03:00
2017-03-24 05:48:14 +03:00
Copy the TLS certificates to the Kubernetes configuration directory:
2016-07-07 17:15:59 +03:00
```
2017-03-24 05:48:14 +03:00
sudo mv ca.pem ca-key.pem kubernetes-key.pem kubernetes.pem /var/lib/kubernetes/
2016-07-07 17:15:59 +03:00
```
2017-03-24 05:48:14 +03:00
### Download and install the Kubernetes controller binaries
Download the official Kubernetes release binaries:
2016-09-11 17:31:04 +03:00
2016-07-07 17:15:59 +03:00
```
2017-03-25 21:41:26 +03:00
wget https://storage.googleapis.com/kubernetes-release/release/v1.6.0-rc.1/bin/linux/amd64/kube-apiserver
2016-07-07 17:15:59 +03:00
```
2017-03-25 21:41:26 +03:00
2016-07-07 17:15:59 +03:00
```
2017-03-25 21:41:26 +03:00
wget https://storage.googleapis.com/kubernetes-release/release/v1.6.0-rc.1/bin/linux/amd64/kube-controller-manager
2017-03-24 05:48:14 +03:00
```
2017-03-25 21:41:26 +03:00
2017-03-24 05:48:14 +03:00
```
2017-03-25 21:41:26 +03:00
wget https://storage.googleapis.com/kubernetes-release/release/v1.6.0-rc.1/bin/linux/amd64/kube-scheduler
2017-03-24 05:48:14 +03:00
```
2017-03-25 21:41:26 +03:00
2017-03-24 05:48:14 +03:00
```
2017-03-25 21:41:26 +03:00
wget https://storage.googleapis.com/kubernetes-release/release/v1.6.0-rc.1/bin/linux/amd64/kubectl
2016-07-07 17:15:59 +03:00
```
2017-03-24 05:48:14 +03:00
Install the Kubernetes binaries:
2016-09-11 17:31:04 +03:00
2016-07-07 17:15:59 +03:00
```
2017-03-24 05:48:14 +03:00
chmod +x kube-apiserver kube-controller-manager kube-scheduler kubectl
2016-07-07 17:15:59 +03:00
```
```
2017-03-24 05:48:14 +03:00
sudo mv kube-apiserver kube-controller-manager kube-scheduler kubectl /usr/bin/
2016-07-07 17:15:59 +03:00
```
2017-03-24 05:48:14 +03:00
### Kubernetes API Server
Capture the internal IP address of the machine:
2016-07-08 20:26:32 +03:00
```
2016-09-11 13:15:28 +03:00
INTERNAL_IP=$(curl -s -H "Metadata-Flavor: Google" \
2016-07-08 20:26:32 +03:00
http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/ip)
2016-07-07 17:15:59 +03:00
```
2016-07-08 20:26:32 +03:00
Create the systemd unit file:
```
2017-03-24 05:48:14 +03:00
cat > kube-apiserver.service <<EOF
2016-07-08 20:26:32 +03:00
[Unit]
2016-07-07 17:15:59 +03:00
Description=Kubernetes API Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
[Service]
2017-03-24 05:48:14 +03:00
ExecStart=/usr/bin/kube-apiserver \\
--admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,ResourceQuota \\
--advertise-address=${INTERNAL_IP} \\
--allow-privileged=true \\
--apiserver-count=3 \\
--audit-log-maxage=30 \\
--audit-log-maxbackup=3 \\
--audit-log-maxsize=100 \\
--audit-log-path=/var/lib/audit.log \\
2017-03-24 05:48:14 +03:00
--authorization-mode=RBAC \\
--bind-address=0.0.0.0 \\
--client-ca-file=/var/lib/kubernetes/ca.pem \\
--enable-swagger-ui=true \\
--etcd-cafile=/var/lib/kubernetes/ca.pem \\
--etcd-certfile=/var/lib/kubernetes/kubernetes.pem \\
--etcd-keyfile=/var/lib/kubernetes/kubernetes-key.pem \\
--etcd-servers=https://10.240.0.10:2379,https://10.240.0.11:2379,https://10.240.0.12:2379 \\
--event-ttl=1h \\
--experimental-bootstrap-token-auth \\
--insecure-bind-address=0.0.0.0 \\
--kubelet-certificate-authority=/var/lib/kubernetes/ca.pem \\
--kubelet-client-certificate=/var/lib/kubernetes/kubernetes.pem \\
--kubelet-client-key=/var/lib/kubernetes/kubernetes-key.pem \\
--kubelet-https=true \\
--runtime-config=rbac.authorization.k8s.io/v1alpha1 \\
2017-03-25 21:41:26 +03:00
--service-account-key-file=/var/lib/kubernetes/ca-key.pem \\
2017-03-24 05:48:14 +03:00
--service-cluster-ip-range=10.32.0.0/24 \\
--service-node-port-range=30000-32767 \\
--tls-cert-file=/var/lib/kubernetes/kubernetes.pem \\
--tls-private-key-file=/var/lib/kubernetes/kubernetes-key.pem \\
--token-auth-file=/var/lib/kubernetes/token.csv \\
2016-07-07 17:15:59 +03:00
--v=2
Restart=on-failure
RestartSec=5
[Install]
2016-07-08 20:26:32 +03:00
WantedBy=multi-user.target
EOF
2016-07-07 17:15:59 +03:00
```
2017-03-24 05:48:14 +03:00
Start the `kube-apiserver` service:
2016-07-07 17:15:59 +03:00
```
2016-07-08 20:26:32 +03:00
sudo mv kube-apiserver.service /etc/systemd/system/
2016-07-07 17:15:59 +03:00
```
2016-07-07 18:37:41 +03:00
```
sudo systemctl daemon-reload
2017-03-25 21:41:26 +03:00
```
```
2016-07-07 18:37:41 +03:00
sudo systemctl enable kube-apiserver
2017-03-25 21:41:26 +03:00
```
```
2016-07-07 18:37:41 +03:00
sudo systemctl start kube-apiserver
```
```
2016-07-08 20:26:32 +03:00
sudo systemctl status kube-apiserver --no-pager
2016-07-07 18:37:41 +03:00
```
2016-07-09 19:28:12 +03:00
### Kubernetes Controller Manager
2016-07-07 18:37:41 +03:00
```
2017-03-24 05:48:14 +03:00
cat > kube-controller-manager.service <<EOF
2016-07-08 20:26:32 +03:00
[Unit]
2016-07-07 18:37:41 +03:00
Description=Kubernetes Controller Manager
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
[Service]
2017-03-24 05:48:14 +03:00
ExecStart=/usr/bin/kube-controller-manager \\
--address=0.0.0.0 \\
--allocate-node-cidrs=true \\
--cluster-cidr=10.200.0.0/16 \\
--cluster-name=kubernetes \\
--cluster-signing-cert-file=/var/lib/kubernetes/ca.pem \\
--cluster-signing-key-file=/var/lib/kubernetes/ca-key.pem \\
2017-03-24 05:48:14 +03:00
--leader-elect=true \\
--master=http://${INTERNAL_IP}:8080 \\
--root-ca-file=/var/lib/kubernetes/ca.pem \\
--service-account-private-key-file=/var/lib/kubernetes/ca-key.pem \\
--service-cluster-ip-range=10.32.0.0/16 \\
2016-07-07 18:37:41 +03:00
--v=2
Restart=on-failure
RestartSec=5
[Install]
2016-07-08 20:26:32 +03:00
WantedBy=multi-user.target
EOF
2016-07-07 18:37:41 +03:00
```
2017-03-24 05:48:14 +03:00
Start the `kube-controller-manager` service:
2016-07-07 18:37:41 +03:00
```
2016-07-08 20:26:32 +03:00
sudo mv kube-controller-manager.service /etc/systemd/system/
2016-07-07 18:37:41 +03:00
```
```
sudo systemctl daemon-reload
2017-03-25 21:41:26 +03:00
```
```
2016-07-08 20:26:32 +03:00
sudo systemctl enable kube-controller-manager
2017-03-25 21:41:26 +03:00
```
```
2016-07-08 20:26:32 +03:00
sudo systemctl start kube-controller-manager
2016-07-07 18:37:41 +03:00
```
```
2016-07-08 20:26:32 +03:00
sudo systemctl status kube-controller-manager --no-pager
2016-07-07 18:37:41 +03:00
```
2016-07-09 19:28:12 +03:00
### Kubernetes Scheduler
2016-07-07 18:37:41 +03:00
```
2017-03-24 05:48:14 +03:00
cat > kube-scheduler.service <<EOF
2016-07-08 20:26:32 +03:00
[Unit]
Description=Kubernetes Scheduler
2016-07-07 18:37:41 +03:00
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
[Service]
2017-03-24 05:48:14 +03:00
ExecStart=/usr/bin/kube-scheduler \\
--leader-elect=true \\
--master=http://${INTERNAL_IP}:8080 \\
2016-07-07 18:37:41 +03:00
--v=2
Restart=on-failure
RestartSec=5
[Install]
2016-07-08 20:26:32 +03:00
WantedBy=multi-user.target
EOF
2016-07-07 18:37:41 +03:00
```
2017-03-24 05:48:14 +03:00
Start the `kube-scheduler` service:
2016-07-07 18:37:41 +03:00
```
2016-07-08 20:26:32 +03:00
sudo mv kube-scheduler.service /etc/systemd/system/
2016-07-07 18:37:41 +03:00
```
```
sudo systemctl daemon-reload
2017-03-25 21:41:26 +03:00
```
```
2016-07-07 18:37:41 +03:00
sudo systemctl enable kube-scheduler
2017-03-25 21:41:26 +03:00
```
```
2016-07-07 18:37:41 +03:00
sudo systemctl start kube-scheduler
```
```
2016-07-08 20:26:32 +03:00
sudo systemctl status kube-scheduler --no-pager
2016-07-07 18:37:41 +03:00
```
2016-07-09 19:28:12 +03:00
### Verification
2016-07-07 18:37:41 +03:00
```
kubectl get componentstatuses
```
2017-03-25 21:41:26 +03:00
2016-07-07 18:37:41 +03:00
```
NAME STATUS MESSAGE ERROR
controller-manager Healthy ok
scheduler Healthy ok
etcd-0 Healthy {"health": "true"}
etcd-1 Healthy {"health": "true"}
2016-07-07 18:37:41 +03:00
etcd-2 Healthy {"health": "true"}
2016-07-07 23:26:27 +03:00
```
2016-07-09 03:26:43 +03:00
> Remember to run these steps on `controller0`, `controller1`, and `controller2`
2016-07-07 23:39:33 +03:00
## Setup Kubernetes API Server Frontend Load Balancer
2016-07-07 23:26:27 +03:00
2017-03-26 00:20:31 +03:00
The virtual machines created in this tutorial will not have permission to complete this section. Run the following commands from the same place used to create the virtual machines for this tutorial.
2016-09-11 08:57:20 +03:00
2016-07-07 23:26:27 +03:00
```
2017-03-25 21:41:26 +03:00
gcloud compute http-health-checks create kube-apiserver-health-check \
2016-07-07 23:26:27 +03:00
--description "Kubernetes API Server Health Check" \
--port 8080 \
--request-path /healthz
```
```
2017-03-25 21:41:26 +03:00
gcloud compute target-pools create kubernetes-target-pool \
--http-health-check=kube-apiserver-health-check
2016-07-07 23:26:27 +03:00
```
```
2017-03-25 21:41:26 +03:00
gcloud compute target-pools add-instances kubernetes-target-pool \
2016-07-08 20:26:32 +03:00
--instances controller0,controller1,controller2
2016-07-07 23:26:27 +03:00
```
2016-07-08 20:28:27 +03:00
```
2017-03-25 21:41:26 +03:00
KUBERNETES_PUBLIC_ADDRESS=$(gcloud compute addresses describe kubernetes-the-hard-way \
--region us-central1 \
2016-07-09 03:33:39 +03:00
--format 'value(address)')
2016-07-08 20:28:27 +03:00
```
2016-07-07 23:26:27 +03:00
```
2017-03-25 21:41:26 +03:00
gcloud compute forwarding-rules create kubernetes-forwarding-rule \
2016-09-11 13:15:28 +03:00
--address ${KUBERNETES_PUBLIC_ADDRESS} \
2016-07-07 23:26:27 +03:00
--ports 6443 \
2017-03-25 21:41:26 +03:00
--target-pool kubernetes-target-pool \
2017-03-24 09:08:54 +03:00
--region us-central1
```