mirrors
/
kubernetes-the-hard-way
mirror of https://github.com/kelseyhightower/kubernetes-the-hard-way
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Merge pull request #1 from smallstep/step-ca 

Kubernetes The Hard Way, With step-ca
pull/707/merge^2
Carl Tashian 2022-05-31 10:19:47 -07:00 committed by GitHub
parent 79a3f79b27 340050478d
commit 8935e47a4f
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
8 changed files with 602 additions and 302 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

58
docs/02-client-tools.md
View File

@ -1,72 +1,56 @@
# Installing the Client Tools # Installing the Client Tools
In this lab you will install the command line utilities required to complete this tutorial: [cfssl](https://github.com/cloudflare/cfssl), [cfssljson](https://github.com/cloudflare/cfssl), and [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl). In this lab you will install the command line utilities required to complete this tutorial: [step](https://github.com/smallstep/cli), and [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl).
## Install CFSSL ## Install CFSSL
The `cfssl` and `cfssljson` command line utilities will be used to provision a [PKI Infrastructure](https://en.wikipedia.org/wiki/Public_key_infrastructure) and generate TLS certificates. The `step` command line utility will be used to provision a [PKI Infrastructure](https://en.wikipedia.org/wiki/Public_key_infrastructure) and generate TLS certificates.
Download and install `cfssl` and `cfssljson`: Download and install `step`:
### OS X ### OS X
``` For Intel chips:
curl -o cfssl https://storage.googleapis.com/kubernetes-the-hard-way/cfssl/1.4.1/darwin/cfssl
curl -o cfssljson https://storage.googleapis.com/kubernetes-the-hard-way/cfssl/1.4.1/darwin/cfssljson
```
``` ```
chmod +x cfssl cfssljson curl -L https://dl.step.sm/gh-release/cli/gh-release-header/v0.18.0/step_darwin_0.18.0_amd64.tar.gz | tar xz
sudo mv step_0.18.0/bin/step /usr/local/bin/
``` ```
``` For Apple Silicon:
sudo mv cfssl cfssljson /usr/local/bin/
```
Some OS X users may experience problems using the pre-built binaries in which case [Homebrew](https://brew.sh) might be a better option:
``` ```
brew install cfssl curl -L https://dl.step.sm/gh-release/cli/gh-release-header/v0.18.0/step_darwin_0.18.0_arm64.tar.gz | tar xz
sudo mv step_0.18.0/bin/step /usr/local/bin/
```
Or, if you'd like to use [Homebrew](https://brew.sh):
```
brew install step
``` ```
### Linux ### Linux
``` ```
wget -q --show-progress --https-only --timestamping \ curl -L https://dl.step.sm/gh-release/cli/gh-release-header/v0.18.0/step_linux_0.18.0_amd64.tar.gz
https://storage.googleapis.com/kubernetes-the-hard-way/cfssl/1.4.1/linux/cfssl \ sudo mv step_0.18.0/bin/step /usr/local/bin/
https://storage.googleapis.com/kubernetes-the-hard-way/cfssl/1.4.1/linux/cfssljson
```
```
chmod +x cfssl cfssljson
```
```
sudo mv cfssl cfssljson /usr/local/bin/
``` ```
### Verification ### Verification
Verify `cfssl` and `cfssljson` version 1.4.1 or higher is installed: Verify `step` version 0.18.0 or higher is installed:
``` ```
cfssl version step version
``` ```
> output > output
``` ```
Version: 1.4.1 Smallstep CLI/0.18.0 (linux/amd64)
Runtime: go1.12.12 Release Date: 2021-11-17 21:15 UTC
```
```
cfssljson --version
```
```
Version: 1.4.1
Runtime: go1.12.12
``` ```
## Install kubectl ## Install kubectl

4
docs/03-compute-resources.md
View File

@ -43,11 +43,11 @@ gcloud compute firewall-rules create kubernetes-the-hard-way-allow-internal \
--source-ranges 10.240.0.0/24,10.200.0.0/16 --source-ranges 10.240.0.0/24,10.200.0.0/16
``` ```
Create a firewall rule that allows external SSH, ICMP, and HTTPS: Create a firewall rule that allows external SSH, ICMP, HTTPS, and step-ca traffic:
``` ```
gcloud compute firewall-rules create kubernetes-the-hard-way-allow-external \ gcloud compute firewall-rules create kubernetes-the-hard-way-allow-external \
--allow tcp:22,tcp:6443,icmp \ --allow tcp:22,tcp:4443,tcp:6443,icmp \
--network kubernetes-the-hard-way \ --network kubernetes-the-hard-way \
--source-ranges 0.0.0.0/0 --source-ranges 0.0.0.0/0
``` ```

552
docs/04-certificate-authority.md
View File

@ -1,61 +1,265 @@
# Provisioning a CA and Generating TLS Certificates # Provisioning a CA
In this lab you will provision a [PKI Infrastructure](https://en.wikipedia.org/wiki/Public_key_infrastructure) using CloudFlare's PKI toolkit, [cfssl](https://github.com/cloudflare/cfssl), then use it to bootstrap a Certificate Authority, and generate TLS certificates for the following components: etcd, kube-apiserver, kube-controller-manager, kube-scheduler, kubelet, and kube-proxy. In this lab you will provision a [PKI Infrastructure](https://en.wikipedia.org/wiki/Public_key_infrastructure) using Smallstep's CA server, [`step-ca`](https://github.com/smallstep/certificates), and generate TLS certificates for the following components: etcd, kube-apiserver, kube-controller-manager, kube-scheduler, kubelet, and kube-proxy.
## Certificate Authority ## Certificate Authority
In this section you will provision a Certificate Authority that can be used to generate additional TLS certificates. In this section you will provision a `step-ca` Certificate Authority that can be used to generate additional TLS certificates. The CA will only run on `controller-0`. While it's possible to run a high-availability CA across multiple nodes, it's not necessary in a small-to-medium sized Kubernetes cluster. The CA service would have to be down for several days before having any negative impact on the cluster.
Generate the CA configuration file, certificate, and private key: Connect to `controller-0`:
```
gcloud compute ssh controller-0
```
Download the `step` client and `step-ca` server binaries, and the `jq` command:
``` ```
{ {
wget -q --show-progress --https-only --timestamping \
"https://dl.step.sm/gh-release/certificates/gh-release-header/v0.18.1/step-ca_linux_0.18.1_amd64.tar.gz" \
"https://dl.step.sm/gh-release/cli/gh-release-header/v0.18.1/step_linux_0.18.1_amd64.tar.gz"
sudo apt update
sudo apt install -y jq
}
```
cat > ca-config.json <<EOF Install the binaries:
```
{ {
"signing": { tar -xvf step-ca_linux_0.18.1_amd64.tar.gz
"default": { sudo mv step-ca_0.18.1/bin/step-ca /usr/local/bin/
"expiry": "8760h" tar -xvf step_linux_0.18.1_amd64.tar.gz
sudo mv step_0.18.1/bin/step /usr/local/bin/
}
```
Now create a `step` user and the paths for `step-ca`:
```
sudo useradd --system --home /etc/step-ca --shell /bin/false step
```
Create a CA configuration folder and generate passwords for the CA root key and the CA provisioner:
```
{
export STEPPATH=/etc/step-ca
umask 077
< /dev/urandom tr -dc A-Za-z0-9 | head -c40 | sudo tee $(step path)/password > /dev/null
< /dev/urandom tr -dc A-Za-z0-9 | head -c40 > provisioner-password
umask 002
}
```
Initialize your PKI:
```
{
INTERNAL_IP=$(curl -s -H "Metadata-Flavor: Google" \
http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/ip)
EXTERNAL_IP=$(curl -s -H "Metadata-Flavor: Google" http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip)
INTERNAL_HOSTNAME=$(hostname -f)
sudo -E step ca init --name="admin" \
--dns="$INTERNAL_IP,$INTERNAL_HOSTNAME,$EXTERNAL_IP" \
--address=":4443" --provisioner="kubernetes" \
--password-file="$(step path)/password" \
--provisioner-password-file="provisioner-password"
}
```
Add an X509 certificate template file:
```
mkdir -p /etc/step-ca/templates/x509
# Server cert template.
cat <<EOF > /etc/step-ca/templates/x509/kubernetes.tpl
{
"subject": {
{{- if .Insecure.User.Organization }}
"organization": {{ toJson .Insecure.User.Organization }},
{{- end }}
"commonName": {{ toJson .Subject.CommonName }},
"organizationalUnit": {{ toJson .OrganizationalUnit }}
}, },
"profiles": { "sans": {{ toJson .SANs }},
"kubernetes": { {{- if typeIs "*rsa.PublicKey" .Insecure.CR.PublicKey }}
"usages": ["signing", "key encipherment", "server auth", "client auth"], "keyUsage": ["keyEncipherment", "digitalSignature"],
"expiry": "8760h" {{- else }}
} "keyUsage": ["digitalSignature"],
} {{- end }}
} "extKeyUsage": ["serverAuth", "clientAuth"]
} }
EOF EOF
```
cat > ca-csr.json <<EOF Configure the CA provisioner to issue 90-day certificates:
```
{ {
"CN": "Kubernetes", cat <<< $(jq '(.authority.provisioners[] | select(.name == "kubernetes")) += {
"key": { "claims": {
"algo": "rsa", "maxTLSCertDuration": "2160h",
"size": 2048 "defaultTLSCertDuration": "2160h"
}, },
"names": [ "options": {
{ "x509": {
"C": "US", "templateFile": "templates/x509/kubernetes.tpl",
"L": "Portland", "templateData": {
"O": "Kubernetes", "OrganizationalUnit": "Kubernetes The Hard Way"
"OU": "CA",
"ST": "Oregon"
} }
] }
}
}' /etc/step-ca/config/ca.json) > /etc/step-ca/config/ca.json
} }
```
Put the CA configuration into place, and add the CA to systemd:
```
{
sudo chown -R step:step /etc/step-ca
cat <<EOF | sudo tee /etc/systemd/system/step-ca.service
[Unit]
Description=step-ca service
Documentation=https://smallstep.com/docs/step-ca
Documentation=https://smallstep.com/docs/step-ca/certificate-authority-server-production
After=network-online.target
Wants=network-online.target
StartLimitIntervalSec=30
StartLimitBurst=3
ConditionFileNotEmpty=/etc/step-ca/config/ca.json
ConditionFileNotEmpty=/etc/step-ca/password
[Service]
Type=simple
User=step
Group=step
Environment=STEPPATH=/etc/step-ca
WorkingDirectory=/etc/step-ca
ExecStart=/usr/local/bin/step-ca config/ca.json --password-file password
ExecReload=/bin/kill --signal HUP $MAINPID
Restart=on-failure
RestartSec=5
TimeoutStopSec=30
StartLimitInterval=30
StartLimitBurst=3
; Process capabilities & privileges
AmbientCapabilities=CAP_NET_BIND_SERVICE
CapabilityBoundingSet=CAP_NET_BIND_SERVICE
SecureBits=keep-caps
NoNewPrivileges=yes
; Sandboxing
ProtectSystem=full
ProtectHome=true
RestrictNamespaces=true
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
PrivateTmp=true
ProtectClock=true
ProtectControlGroups=true
ProtectKernelTunables=true
ProtectKernelLogs=true
ProtectKernelModules=true
LockPersonality=true
RestrictSUIDSGID=true
RemoveIPC=true
RestrictRealtime=true
PrivateDevices=true
SystemCallFilter=@system-service
SystemCallArchitectures=native
MemoryDenyWriteExecute=true
ReadWriteDirectories=/etc/step-ca/db
[Install]
WantedBy=multi-user.target
EOF EOF
cfssl gencert -initca ca-csr.json | cfssljson -bare ca
} }
``` ```
Results: Save the root CA certificate:
```
sudo cat /etc/step-ca/certs/root_ca.crt | tee ca.pem > /dev/null
```
Finally, start the CA service:
```
{
sudo systemctl daemon-reload
sudo systemctl enable --now step-ca
}
```
## Verification
Check the CA health, then request and save the CA root certificate:
```
sudo -u step -E step ca health
```
Output:
```
ok
```
You can now sign out of `controller-0`.
# Generating certificates
## Bootstrapping with the CA
### Bootstrapping your local machine
Run the following on your local machine.
Download your CA's root certificate:
```
gcloud compute scp controller-0:ca.pem controller-0:provisioner-password .
```
Result:
``` ```
ca-key.pem
ca.pem ca.pem
provisioner-password
```
Now bootstrap with your CA:
```
{
CA_IP=$(gcloud compute instances describe controller-0 \
--format='get(networkInterfaces[0].accessConfigs[0].natIP)')
step ca bootstrap --ca-url "https://$CA_IP:4443/" --fingerprint $(step certificate fingerprint ca.pem)
}
```
Output:
```
The root certificate has been saved in /home/carl/.step/authorities/XX.XXX.XXX.XXX/certs/root_ca.crt.
The authority configuration has been saved in /home/carl/.step/authorities/XX.XXX.XXX.XXX/config/defaults.json.
The profile configuration has been saved in /home/carl/.step/profiles/XX.XXX.XXX.XXX/config/defaults.json.
```
Add your CA URL and fingerprint to the project metadata on GCP, so instances can bootstrap:
```
gcloud compute project-info add-metadata --metadata="STEP_CA_URL=https://10.240.0.10:4443,STEP_CA_FINGERPRINT=$(step certificate fingerprint ca.pem)"
```
Output:
```
Updated [https://www.googleapis.com/compute/v1/projects/project-id-xxxxxx].
``` ```
## Client and Server Certificates ## Client and Server Certificates
@ -64,37 +268,15 @@ In this section you will generate client and server certificates for each Kubern
### The Admin Client Certificate ### The Admin Client Certificate
Generate the `admin` client certificate and private key: On your local machine, generate the `admin` client certificate and private key:
``` ```
{ {
step ca certificate admin admin.pem admin-key.pem \
cat > admin-csr.json <<EOF --provisioner="kubernetes" \
{ --provisioner-password-file="provisioner-password" \
"CN": "admin", --set "Organization=system:masters" \
"key": { --kty RSA
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "US",
"L": "Portland",
"O": "system:masters",
"OU": "Kubernetes The Hard Way",
"ST": "Oregon"
}
]
}
EOF
cfssl gencert \
-ca=ca.pem \
-ca-key=ca-key.pem \
-config=ca-config.json \
-profile=kubernetes \
admin-csr.json | cfssljson -bare admin
} }
``` ```
@ -113,24 +295,6 @@ Generate a certificate and private key for each Kubernetes worker node:
``` ```
for instance in worker-0 worker-1 worker-2; do for instance in worker-0 worker-1 worker-2; do
cat > ${instance}-csr.json <<EOF
{
"CN": "system:node:${instance}",
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "US",
"L": "Portland",
"O": "system:nodes",
"OU": "Kubernetes The Hard Way",
"ST": "Oregon"
}
]
}
EOF
EXTERNAL_IP=$(gcloud compute instances describe ${instance} \ EXTERNAL_IP=$(gcloud compute instances describe ${instance} \
--format 'value(networkInterfaces[0].accessConfigs[0].natIP)') --format 'value(networkInterfaces[0].accessConfigs[0].natIP)')
@ -138,13 +302,13 @@ EXTERNAL_IP=$(gcloud compute instances describe ${instance} \
INTERNAL_IP=$(gcloud compute instances describe ${instance} \ INTERNAL_IP=$(gcloud compute instances describe ${instance} \
--format 'value(networkInterfaces[0].networkIP)') --format 'value(networkInterfaces[0].networkIP)')
cfssl gencert \ step ca certificate "system:node:${instance}" ${instance}.pem ${instance}-key.pem \
-ca=ca.pem \ --san "${instance}" \
-ca-key=ca-key.pem \ --san "${EXTERNAL_IP}" \
-config=ca-config.json \ --san "${INTERNAL_IP}" \
-hostname=${instance},${EXTERNAL_IP},${INTERNAL_IP} \ --set "Organization=system:nodes" \
-profile=kubernetes \ --provisioner "kubernetes" \
${instance}-csr.json | cfssljson -bare ${instance} --provisioner-password-file "provisioner-password"
done done
``` ```
@ -161,37 +325,25 @@ worker-2.pem
### The Controller Manager Client Certificate ### The Controller Manager Client Certificate
Generate the `kube-controller-manager` client certificate and private key: Generate the `kube-controller-manager`, `kube-proxy`, and `kube-scheduler` client certificates and private keys:
``` ```
{ {
step ca certificate "system:kube-controller-manager" kube-controller-manager.pem kube-controller-manager-key.pem \
cat > kube-controller-manager-csr.json <<EOF --kty RSA \
{ --set "Organization=system:kube-controller-manager" \
"CN": "system:kube-controller-manager", --provisioner "kubernetes" \
"key": { --provisioner-password-file "provisioner-password"
"algo": "rsa", step ca certificate "system:kube-proxy" kube-proxy.pem kube-proxy-key.pem \
"size": 2048 --kty RSA \
}, --set "Organization=system:node-proxier" \
"names": [ --provisioner "kubernetes" \
{ --provisioner-password-file "provisioner-password"
"C": "US", step ca certificate "system:kube-scheduler" kube-scheduler.pem kube-scheduler-key.pem \
"L": "Portland", --kty RSA \
"O": "system:kube-controller-manager", --set "Organization=system:kube-scheduler" \
"OU": "Kubernetes The Hard Way", --provisioner "kubernetes" \
"ST": "Oregon" --provisioner-password-file "provisioner-password"
}
]
}
EOF
cfssl gencert \
-ca=ca.pem \
-ca-key=ca-key.pem \
-config=ca-config.json \
-profile=kubernetes \
kube-controller-manager-csr.json | cfssljson -bare kube-controller-manager
} }
``` ```
@ -200,96 +352,12 @@ Results:
``` ```
kube-controller-manager-key.pem kube-controller-manager-key.pem
kube-controller-manager.pem kube-controller-manager.pem
```
### The Kube Proxy Client Certificate
Generate the `kube-proxy` client certificate and private key:
```
{
cat > kube-proxy-csr.json <<EOF
{
"CN": "system:kube-proxy",
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "US",
"L": "Portland",
"O": "system:node-proxier",
"OU": "Kubernetes The Hard Way",
"ST": "Oregon"
}
]
}
EOF
cfssl gencert \
-ca=ca.pem \
-ca-key=ca-key.pem \
-config=ca-config.json \
-profile=kubernetes \
kube-proxy-csr.json | cfssljson -bare kube-proxy
}
```
Results:
```
kube-proxy-key.pem kube-proxy-key.pem
kube-proxy.pem kube-proxy.pem
```
### The Scheduler Client Certificate
Generate the `kube-scheduler` client certificate and private key:
```
{
cat > kube-scheduler-csr.json <<EOF
{
"CN": "system:kube-scheduler",
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "US",
"L": "Portland",
"O": "system:kube-scheduler",
"OU": "Kubernetes The Hard Way",
"ST": "Oregon"
}
]
}
EOF
cfssl gencert \
-ca=ca.pem \
-ca-key=ca-key.pem \
-config=ca-config.json \
-profile=kubernetes \
kube-scheduler-csr.json | cfssljson -bare kube-scheduler
}
```
Results:
```
kube-scheduler-key.pem kube-scheduler-key.pem
kube-scheduler.pem kube-scheduler.pem
``` ```
### The Kubernetes API Server Certificate ### The Kubernetes API Server Certificate
The `kubernetes-the-hard-way` static IP address will be included in the list of subject alternative names for the Kubernetes API Server certificate. This will ensure the certificate can be validated by remote clients. The `kubernetes-the-hard-way` static IP address will be included in the list of subject alternative names for the Kubernetes API Server certificate. This will ensure the certificate can be validated by remote clients.
@ -298,40 +366,25 @@ Generate the Kubernetes API Server certificate and private key:
``` ```
{ {
KUBERNETES_PUBLIC_ADDRESS=$(gcloud compute addresses describe kubernetes-the-hard-way \ KUBERNETES_PUBLIC_ADDRESS=$(gcloud compute addresses describe kubernetes-the-hard-way \
--region $(gcloud config get-value compute/region) \ --region $(gcloud config get-value compute/region) \
--format 'value(address)') --format 'value(address)')
step ca certificate "kubernetes" kubernetes.pem kubernetes-key.pem \
KUBERNETES_HOSTNAMES=kubernetes,kubernetes.default,kubernetes.default.svc,kubernetes.default.svc.cluster,kubernetes.svc.cluster.local --kty RSA \
--san kubernetes \
cat > kubernetes-csr.json <<EOF --san kubernetes.default \
{ --san kubernetes.default.svc \
"CN": "kubernetes", --san kubernetes.default.svc.cluster \
"key": { --san kubernetes.default.svc.cluster.local \
"algo": "rsa", --san 10.32.0.1 \
"size": 2048 --san 10.240.0.10 \
}, --san 10.240.0.11 \
"names": [ --san 10.240.0.12 \
{ --san ${KUBERNETES_PUBLIC_ADDRESS} \
"C": "US", --san 127.0.0.1 \
"L": "Portland", --set "Organization=Kubernetes" \
"O": "Kubernetes", --provisioner "kubernetes" \
"OU": "Kubernetes The Hard Way", --provisioner-password-file "provisioner-password"
"ST": "Oregon"
}
]
}
EOF
cfssl gencert \
-ca=ca.pem \
-ca-key=ca-key.pem \
-config=ca-config.json \
-hostname=10.32.0.1,10.240.0.10,10.240.0.11,10.240.0.12,${KUBERNETES_PUBLIC_ADDRESS},127.0.0.1,${KUBERNETES_HOSTNAMES} \
-profile=kubernetes \
kubernetes-csr.json | cfssljson -bare kubernetes
} }
``` ```
@ -352,33 +405,11 @@ Generate the `service-account` certificate and private key:
``` ```
{ {
step ca certificate "service-accounts" service-account.pem service-account-key.pem \
cat > service-account-csr.json <<EOF --kty RSA \
{ --set "Organization=Kubernetes" \
"CN": "service-accounts", --provisioner "kubernetes" \
"key": { --provisioner-password-file "provisioner-password"
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "US",
"L": "Portland",
"O": "Kubernetes",
"OU": "Kubernetes The Hard Way",
"ST": "Oregon"
}
]
}
EOF
cfssl gencert \
-ca=ca.pem \
-ca-key=ca-key.pem \
-config=ca-config.json \
-profile=kubernetes \
service-account-csr.json | cfssljson -bare service-account
} }
``` ```
@ -396,7 +427,8 @@ Copy the appropriate certificates and private keys to each worker instance:
``` ```
for instance in worker-0 worker-1 worker-2; do for instance in worker-0 worker-1 worker-2; do
gcloud compute scp ca.pem ${instance}-key.pem ${instance}.pem ${instance}:~/ gcloud compute scp ca.pem ${instance}-key.pem ${instance}.pem \
kube-proxy-key.pem kube-proxy.pem ${instance}:~/
done done
``` ```
@ -404,8 +436,10 @@ Copy the appropriate certificates and private keys to each controller instance:
``` ```
for instance in controller-0 controller-1 controller-2; do for instance in controller-0 controller-1 controller-2; do
gcloud compute scp ca.pem ca-key.pem kubernetes-key.pem kubernetes.pem \ gcloud compute scp ca.pem kubernetes-key.pem kubernetes.pem \
service-account-key.pem service-account.pem ${instance}:~/ service-account-key.pem service-account.pem \
kube-controller-manager-key.pem kube-controller-manager.pem \
kube-scheduler-key.pem kube-scheduler.pem ${instance}:~/
done done
``` ```

6
docs/08-bootstrapping-kubernetes-controllers.md
View File

@ -49,8 +49,11 @@ Install the Kubernetes binaries:
{ {
sudo mkdir -p /var/lib/kubernetes/ sudo mkdir -p /var/lib/kubernetes/
sudo mv ca.pem ca-key.pem kubernetes-key.pem kubernetes.pem \ sudo mv ca.pem kubernetes-key.pem kubernetes.pem \
service-account-key.pem service-account.pem \ service-account-key.pem service-account.pem \
kube-proxy.pem kube-proxy-key.pem \
kube-controller-manager.pem kube-controller-manager-key.pem \
kube-scheduler.pem kube-scheduler-key.pem \
encryption-config.yaml /var/lib/kubernetes/ encryption-config.yaml /var/lib/kubernetes/
} }
``` ```
@ -142,7 +145,6 @@ ExecStart=/usr/local/bin/kube-controller-manager \\
--cluster-cidr=10.200.0.0/16 \\ --cluster-cidr=10.200.0.0/16 \\
--cluster-name=kubernetes \\ --cluster-name=kubernetes \\
--cluster-signing-cert-file=/var/lib/kubernetes/ca.pem \\ --cluster-signing-cert-file=/var/lib/kubernetes/ca.pem \\
--cluster-signing-key-file=/var/lib/kubernetes/ca-key.pem \\
--kubeconfig=/var/lib/kubernetes/kube-controller-manager.kubeconfig \\ --kubeconfig=/var/lib/kubernetes/kube-controller-manager.kubeconfig \\
--leader-elect=true \\ --leader-elect=true \\
--root-ca-file=/var/lib/kubernetes/ca.pem \\ --root-ca-file=/var/lib/kubernetes/ca.pem \\

1
docs/09-bootstrapping-kubernetes-workers.md
View File

@ -244,6 +244,7 @@ EOF
``` ```
sudo mv kube-proxy.kubeconfig /var/lib/kube-proxy/kubeconfig sudo mv kube-proxy.kubeconfig /var/lib/kube-proxy/kubeconfig
sudo mv kube-proxy.pem kube-proxy-key.pem /var/lib/kube-proxy
``` ```
Create the `kube-proxy-config.yaml` configuration file: Create the `kube-proxy-config.yaml` configuration file:

279
docs/13-certificate-renewal.md Normal file
View File

@ -0,0 +1,279 @@
# Configuring Certificate Renewal
## Prerequisites
The commands in this section must be run on every instance: `controller-0`, `controller-1`, `controller-2`, `worker-0`, `worker-1`, and `worker-2`. Login to each instance using the `gcloud` command. Example:
```
gcloud compute ssh controller-0
```
## Download certificate management tools
Run each command on every node.
Download the `step` CLI binary:
```
wget -q --show-progress --https-only --timestamping \
"https://dl.step.sm/gh-release/cli/gh-release-header/v0.18.1/step_linux_0.18.1_amd64.tar.gz"
```
Install the binary:
```
tar -xvf step_linux_0.18.1_amd64.tar.gz
sudo mv step_0.18.1/bin/step /usr/local/bin/
```
## Bootstrap with the CA
Configure the host to trust your Certificate Authority:
```
{
STEP_CA_URL=$(gcloud compute project-info describe --format='get(commonInstanceMetadata.items.STEP_CA_URL)')
STEP_CA_FINGERPRINT=$(gcloud compute project-info describe --format='get(commonInstanceMetadata.items.STEP_CA_FINGERPRINT)')
sudo step ca bootstrap \
--ca-url "${STEP_CA_URL}" \
--fingerprint "${STEP_CA_FINGERPRINT}"
}
```
Output:
```
The root certificate has been saved in /root/.step/certs/root_ca.crt.
The authority configuration has been saved in /root/.step/config/defaults.json.
```
## Set up the certificate renewal timer
We'll use a systemd timer to renew certificates when they are 2/3rds of the way through their validity period.
Install the systemd certificate renewal service and timer.
```
cat << EOF | sudo tee /etc/systemd/system/cert-renewer@.service
[Unit]
Description=Certificate renewer for %I
After=network-online.target
Documentation=https://smallstep.com/docs/step-ca/certificate-authority-server-production
StartLimitIntervalSec=0
[Service]
Type=oneshot
User=root
Environment=STEPPATH=/etc/step-ca \\
CERT_LOCATION=/etc/step/certs/%i.crt \\
KEY_LOCATION=/etc/step/certs/%i.key
; ExecCondition checks if the certificate is ready for renewal,
; based on the exit status of the command.
; (In systemd <242, you can use ExecStartPre= here.)
ExecCondition=/usr/local/bin/step certificate needs-renewal \${CERT_LOCATION}
; ExecStart renews the certificate, if ExecStartPre was successful.
ExecStart=/usr/local/bin/step ca renew --force \${CERT_LOCATION} \${KEY_LOCATION}
[Install]
WantedBy=multi-user.target
EOF
```
Install the timer:
```
cat << EOF | sudo tee /etc/systemd/system/cert-renewer@.timer
[Unit]
Description=Certificate renewal timer for %I
Documentation=https://smallstep.com/docs/step-ca/certificate-authority-server-production
[Timer]
Persistent=true
; Run the timer unit every 5 minutes.
OnCalendar=*:1/5
; Always run the timer on time.
AccuracySec=1us
; Add jitter to prevent a "thundering hurd" of simultaneous certificate renewals.
RandomizedDelaySec=5m
[Install]
WantedBy=timers.target
EOF
```
# Controller Certificate Renewal
## Prerequisites
The commands in this section must be run on every controller: `controller-0`, `controller-1`, `controller-2`. Login to each instance using the `gcloud` command. Example:
```
gcloud compute ssh controller-0
```
## Configure certificate renewal for etcd
Create and start a certificate renewal timer for etcd:
```
sudo mkdir /etc/systemd/system/cert-renewer@etcd.service.d
cat <<EOF | sudo tee /etc/systemd/system/cert-renewer@etcd.service.d/override.conf
[Service]
Environment=STEPPATH=/root/.step \\
CERT_LOCATION=/etc/etcd/kubernetes.pem \\
KEY_LOCATION=/etc/etcd/kubernetes-key.pem
EOF
sudo systemctl daemon-reload
sudo systemctl enable --now cert-renewer@etcd.timer
```
## Configure certificate renewal for `kube-controller-manager`
Create and start a certificate renewal timer for `kube-controller-manager`. This one will use `kubectl` to embed the renewed certificate and key into the kubeconfig file before restarting the controller manager. Run:
```
sudo mkdir /etc/systemd/system/cert-renewer@kube-controller-manager.service.d
cat <<EOF | sudo tee /etc/systemd/system/cert-renewer@kube-controller-manager.service.d/override.conf
[Service]
Environment=STEPPATH=/root/.step \\
CERT_LOCATION=/var/lib/kubernetes/kube-controller-manager.pem \\
KEY_LOCATION=/var/lib/kubernetes/kube-controller-manager-key.pem
ExecStartPost=kubectl config set-credentials system:kube-controller-manager \\
--client-certificate=\${CERT_LOCATION} \\
--client-key=\${KEY_LOCATION} \\
--embed-certs=true \\
--kubeconfig=/var/lib/kubernetes/kube-controller-manager.kubeconfig
ExecStartPost=systemctl restart kube-controller-manager.service
EOF
sudo systemctl daemon-reload
sudo systemctl enable --now cert-renewer@kube-controller-manager.timer
```
## Configure certificate renewal for kube-scheduler
Create and start a certificate renewal timer for `kube-scheduler`:
```
sudo mkdir /etc/systemd/system/cert-renewer@kube-scheduler.service.d
cat <<EOF | sudo tee /etc/systemd/system/cert-renewer@kube-scheduler.service.d/override.conf
[Service]
Environment=STEPPATH=/root/.step \\
CERT_LOCATION=/var/lib/kubernetes/kube-scheduler.pem \\
KEY_LOCATION=/var/lib/kubernetes/kube-scheduler-key.pem
ExecStartPost=kubectl config set-credentials system:kube-scheduler \\
--client-certificate=\${CERT_LOCATION} \\
--client-key=\${KEY_LOCATION} \\
--embed-certs=true \\
--kubeconfig=/var/lib/kubernetes/kube-scheduler.kubeconfig
ExecStartPost=systemctl restart kube-scheduler.service
EOF
sudo systemctl daemon-reload
sudo systemctl enable --now cert-renewer@kube-scheduler.timer
```
## Configure certificate renewal for kube-apiserver
Create and start a certificate renewal timer for `kube-apiserver`:
```
sudo mkdir /etc/systemd/system/cert-renewer@kube-apiserver.service.d
cat <<EOF | sudo tee /etc/systemd/system/cert-renewer@kube-apiserver.service.d/override.conf
[Service]
Environment=STEPPATH=/root/.step \\
CERT_LOCATION=/var/lib/kubernetes/kubernetes.pem \\
KEY_LOCATION=/var/lib/kubernetes/kubernetes-key.pem
ExecStartPost=systemctl restart kube-apiserver.service
EOF
sudo systemctl daemon-reload
sudo systemctl enable --now cert-renewer@kube-apiserver.timer
```
## Configure service account certificate renewal timer
The service account certificate and key is used by the API server, so we will need to restart it when the certificate file is updated:
```
sudo mkdir /etc/systemd/system/cert-renewer@kube-service-account.service.d
cat <<EOF | sudo tee /etc/systemd/system/cert-renewer@kube-service-account.service.d/override.conf
[Service]
Environment=STEPPATH=/root/.step \\
CERT_LOCATION=/var/lib/kubernetes/service-account.pem \\
KEY_LOCATION=/var/lib/kubernetes/service-account-key.pem
; Restart services that use the service account certificate or key
ExecStartPost=systemctl restart kube-apiserver.service
EOF
sudo systemctl daemon-reload
sudo systemctl enable --now cert-renewer@kube-service-account.timer
```
> Remember to run the above commands on each controller node: `controller-0`, `controller-1`, and `controller-2`.
# Worker Certificate Renewal
## Prerequisites
The commands in this section must be run on every worker: `worker-0`, `worker-1`, and `worker-2`. Login to each instance using the `gcloud` command. Example:
```
gcloud compute ssh worker-0
```
## Configure Certificate Renewal for `kubelet.service`
Install the a renewal service that will restart `kubelet.service` when the certificate is renewed:
```
sudo mkdir /etc/systemd/system/cert-renewer@kubelet.service.d
cat <<EOF | sudo tee /etc/systemd/system/cert-renewer@kubelet.service.d/override.conf
[Service]
Environment=STEPPATH=/root/.step \\
CERT_LOCATION=/var/lib/kubelet/${HOSTNAME}.pem \\
KEY_LOCATION=/var/lib/kubelet/${HOSTNAME}-key.pem
; Restart services that use the service account certificate or key
ExecStartPost=systemctl restart kubelet.service
EOF
sudo systemctl daemon-reload
sudo systemctl enable --now cert-renewer@kubelet.timer
```
## Configure Certificate Renewal for `kube-proxy.service`
Install a renewal service that will rebuild the kubeconfig file and restart kube-proxy when the certificate is renewed:
```
sudo mkdir /etc/systemd/system/cert-renewer@kube-proxy.service.d
cat <<EOF | sudo tee /etc/systemd/system/cert-renewer@kube-proxy.service.d/override.conf
[Service]
Environment=STEPPATH=/root/.step \\
CERT_LOCATION=/var/lib/kube-proxy/kube-proxy.pem \\
KEY_LOCATION=/var/lib/kube-proxy/kube-proxy.pem
ExecStartPost=kubectl config set-credentials system:kube-proxy \\
--client-certificate=\${CERT_LOCATION} \\
--client-key=\${KEY_LOCATION} \\
--embed-certs=true \\
--kubeconfig=/var/lib/kube-proxy/kubeconfig
ExecStartPost=systemctl restart kube-proxy.service
EOF
sudo systemctl daemon-reload
sudo systemctl enable --now cert-renewer@kube-proxy.timer
```
> Remember to run the above commands on each controller node: `worker-0`, `worker-1`, and `worker-2`.

0
docs/13-smoke-test.md → docs/14-smoke-test.md
View File

0
docs/14-cleanup.md → docs/15-cleanup.md
View File