266 lines
8.0 KiB
Markdown
266 lines
8.0 KiB
Markdown
# Bootstrapping the Kubernetes Control Plane
|
|
|
|
In this lab you will bootstrap the Kubernetes control plane across three compute instances and configure it for high availability. You will also create an external load balancer that exposes the Kubernetes API Servers to remote clients. The following components will be installed on each node: Kubernetes API Server, Scheduler, and Controller Manager.
|
|
|
|
## Prerequisites
|
|
|
|
The commands in this lab must be run on each controller instance: `controller-0`, `controller-1`, and `controller-2`. Login to each controller instance using the `gcloud` command. Example:
|
|
|
|
```
|
|
gcloud compute ssh controller-0
|
|
```
|
|
|
|
## Provision the Kubernetes Control Plane
|
|
|
|
### Download and Install the Kubernetes Controller Binaries
|
|
|
|
Download the official Kubernetes release binaries:
|
|
|
|
```
|
|
wget -q --show-progress --https-only --timestamping \
|
|
"https://storage.googleapis.com/kubernetes-release/release/v1.7.4/bin/linux/amd64/kube-apiserver" \
|
|
"https://storage.googleapis.com/kubernetes-release/release/v1.7.4/bin/linux/amd64/kube-controller-manager" \
|
|
"https://storage.googleapis.com/kubernetes-release/release/v1.7.4/bin/linux/amd64/kube-scheduler" \
|
|
"https://storage.googleapis.com/kubernetes-release/release/v1.7.4/bin/linux/amd64/kubectl"
|
|
```
|
|
|
|
Install the Kubernetes binaries:
|
|
|
|
```
|
|
chmod +x kube-apiserver kube-controller-manager kube-scheduler kubectl
|
|
```
|
|
|
|
```
|
|
sudo mv kube-apiserver kube-controller-manager kube-scheduler kubectl /usr/local/bin/
|
|
```
|
|
|
|
### Configure the Kubernetes API Server
|
|
|
|
```
|
|
sudo mkdir -p /var/lib/kubernetes/
|
|
```
|
|
|
|
```
|
|
sudo mv ca.pem ca-key.pem kubernetes-key.pem kubernetes.pem encryption-config.yaml /var/lib/kubernetes/
|
|
```
|
|
|
|
The instance internal IP address will be used advertise the API Server to members of the cluster. Retrieve the internal IP address for the current compute instance:
|
|
|
|
```
|
|
INTERNAL_IP=$(curl -s -H "Metadata-Flavor: Google" \
|
|
http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/ip)
|
|
```
|
|
|
|
Create the `kube-apiserver.service` systemd unit file:
|
|
|
|
```
|
|
cat > kube-apiserver.service <<EOF
|
|
[Unit]
|
|
Description=Kubernetes API Server
|
|
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
|
|
|
|
[Service]
|
|
ExecStart=/usr/local/bin/kube-apiserver \\
|
|
--admission-control=NamespaceLifecycle,NodeRestriction,LimitRanger,ServiceAccount,DefaultStorageClass,ResourceQuota \\
|
|
--advertise-address=${INTERNAL_IP} \\
|
|
--allow-privileged=true \\
|
|
--apiserver-count=3 \\
|
|
--audit-log-maxage=30 \\
|
|
--audit-log-maxbackup=3 \\
|
|
--audit-log-maxsize=100 \\
|
|
--audit-log-path=/var/log/audit.log \\
|
|
--authorization-mode=Node,RBAC \\
|
|
--bind-address=0.0.0.0 \\
|
|
--client-ca-file=/var/lib/kubernetes/ca.pem \\
|
|
--enable-swagger-ui=true \\
|
|
--etcd-cafile=/var/lib/kubernetes/ca.pem \\
|
|
--etcd-certfile=/var/lib/kubernetes/kubernetes.pem \\
|
|
--etcd-keyfile=/var/lib/kubernetes/kubernetes-key.pem \\
|
|
--etcd-servers=https://10.240.0.10:2379,https://10.240.0.11:2379,https://10.240.0.12:2379 \\
|
|
--event-ttl=1h \\
|
|
--experimental-encryption-provider-config=/var/lib/kubernetes/encryption-config.yaml \\
|
|
--insecure-bind-address=0.0.0.0 \\
|
|
--kubelet-certificate-authority=/var/lib/kubernetes/ca.pem \\
|
|
--kubelet-client-certificate=/var/lib/kubernetes/kubernetes.pem \\
|
|
--kubelet-client-key=/var/lib/kubernetes/kubernetes-key.pem \\
|
|
--kubelet-https=true \\
|
|
--runtime-config=rbac.authorization.k8s.io/v1alpha1 \\
|
|
--service-account-key-file=/var/lib/kubernetes/ca-key.pem \\
|
|
--service-cluster-ip-range=10.32.0.0/24 \\
|
|
--service-node-port-range=30000-32767 \\
|
|
--tls-ca-file=/var/lib/kubernetes/ca.pem \\
|
|
--tls-cert-file=/var/lib/kubernetes/kubernetes.pem \\
|
|
--tls-private-key-file=/var/lib/kubernetes/kubernetes-key.pem \\
|
|
--v=2
|
|
Restart=on-failure
|
|
RestartSec=5
|
|
|
|
[Install]
|
|
WantedBy=multi-user.target
|
|
EOF
|
|
```
|
|
|
|
### Configure the Kubernetes Controller Manager
|
|
|
|
Create the `kube-controller-manager.service` systemd unit file:
|
|
|
|
```
|
|
cat > kube-controller-manager.service <<EOF
|
|
[Unit]
|
|
Description=Kubernetes Controller Manager
|
|
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
|
|
|
|
[Service]
|
|
ExecStart=/usr/local/bin/kube-controller-manager \\
|
|
--address=0.0.0.0 \\
|
|
--cluster-cidr=10.200.0.0/16 \\
|
|
--cluster-name=kubernetes \\
|
|
--cluster-signing-cert-file=/var/lib/kubernetes/ca.pem \\
|
|
--cluster-signing-key-file=/var/lib/kubernetes/ca-key.pem \\
|
|
--leader-elect=true \\
|
|
--master=http://${INTERNAL_IP}:8080 \\
|
|
--root-ca-file=/var/lib/kubernetes/ca.pem \\
|
|
--service-account-private-key-file=/var/lib/kubernetes/ca-key.pem \\
|
|
--service-cluster-ip-range=10.32.0.0/16 \\
|
|
--v=2
|
|
Restart=on-failure
|
|
RestartSec=5
|
|
|
|
[Install]
|
|
WantedBy=multi-user.target
|
|
EOF
|
|
```
|
|
|
|
### Configure the Kubernetes Scheduler
|
|
|
|
Create the `kube-scheduler.service` systemd unit file:
|
|
|
|
```
|
|
cat > kube-scheduler.service <<EOF
|
|
[Unit]
|
|
Description=Kubernetes Scheduler
|
|
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
|
|
|
|
[Service]
|
|
ExecStart=/usr/local/bin/kube-scheduler \\
|
|
--leader-elect=true \\
|
|
--master=http://${INTERNAL_IP}:8080 \\
|
|
--v=2
|
|
Restart=on-failure
|
|
RestartSec=5
|
|
|
|
[Install]
|
|
WantedBy=multi-user.target
|
|
EOF
|
|
```
|
|
|
|
### Start the Controller Services
|
|
|
|
```
|
|
sudo mv kube-apiserver.service kube-scheduler.service kube-controller-manager.service /etc/systemd/system/
|
|
```
|
|
|
|
```
|
|
sudo systemctl daemon-reload
|
|
```
|
|
|
|
```
|
|
sudo systemctl enable kube-apiserver kube-controller-manager kube-scheduler
|
|
```
|
|
|
|
```
|
|
sudo systemctl start kube-apiserver kube-controller-manager kube-scheduler
|
|
```
|
|
|
|
> Allow up to 10 seconds for the Kubernetes API Server to fully initialize.
|
|
|
|
### Verification
|
|
|
|
```
|
|
kubectl get componentstatuses
|
|
```
|
|
|
|
```
|
|
NAME STATUS MESSAGE ERROR
|
|
controller-manager Healthy ok
|
|
scheduler Healthy ok
|
|
etcd-2 Healthy {"health": "true"}
|
|
etcd-0 Healthy {"health": "true"}
|
|
etcd-1 Healthy {"health": "true"}
|
|
```
|
|
|
|
> Remember to run the above commands on each controller node: `controller-0`, `controller-1`, and `controller-2`.
|
|
|
|
## The Kubernetes Frontend Load Balancer
|
|
|
|
In this section you will provision an external load balancer to front the Kubernetes API Servers. The `kubernetes-the-hard-way` static IP address will be attached to the resulting load balancer.
|
|
|
|
> The compute instances created in this tutorial will not have permission to complete this section. Run the following commands from the same machine used to create the compute instances.
|
|
|
|
Create the external load balancer network resources:
|
|
|
|
```
|
|
gcloud compute http-health-checks create kube-apiserver-health-check \
|
|
--description "Kubernetes API Server Health Check" \
|
|
--port 8080 \
|
|
--request-path /healthz
|
|
```
|
|
|
|
```
|
|
gcloud compute target-pools create kubernetes-target-pool \
|
|
--http-health-check=kube-apiserver-health-check
|
|
```
|
|
|
|
```
|
|
gcloud compute target-pools add-instances kubernetes-target-pool \
|
|
--instances controller-0,controller-1,controller-2
|
|
```
|
|
|
|
```
|
|
KUBERNETES_PUBLIC_ADDRESS=$(gcloud compute addresses describe kubernetes-the-hard-way \
|
|
--region $(gcloud config get-value compute/region) \
|
|
--format 'value(name)')
|
|
```
|
|
|
|
```
|
|
gcloud compute forwarding-rules create kubernetes-forwarding-rule \
|
|
--address ${KUBERNETES_PUBLIC_ADDRESS} \
|
|
--ports 6443 \
|
|
--region $(gcloud config get-value compute/region) \
|
|
--target-pool kubernetes-target-pool
|
|
```
|
|
|
|
### Verification
|
|
|
|
Retrieve the `kubernetes-the-hard-way` static IP address:
|
|
|
|
```
|
|
KUBERNETES_PUBLIC_ADDRESS=$(gcloud compute addresses describe kubernetes-the-hard-way \
|
|
--region $(gcloud config get-value compute/region) \
|
|
--format 'value(address)')
|
|
```
|
|
|
|
Make a HTTP request for the Kubernetes version info:
|
|
|
|
```
|
|
curl --cacert ca.pem https://${KUBERNETES_PUBLIC_ADDRESS}:6443/version
|
|
```
|
|
|
|
> output
|
|
|
|
```
|
|
{
|
|
"major": "1",
|
|
"minor": "7",
|
|
"gitVersion": "v1.7.4",
|
|
"gitCommit": "793658f2d7ca7f064d2bdf606519f9fe1229c381",
|
|
"gitTreeState": "clean",
|
|
"buildDate": "2017-08-17T08:30:51Z",
|
|
"goVersion": "go1.8.3",
|
|
"compiler": "gc",
|
|
"platform": "linux/amd64"
|
|
}
|
|
```
|
|
|
|
Next: [Bootstrapping the Kubernetes Worker Nodes](09-bootstrapping-kubernetes-workers.md)
|