Carl Tashian
GitHub
commit
8935e47a4f
|
|
@ -1,72 +1,56 @@
|
|||
# Installing the Client Tools
|
||||
|
||||
In this lab you will install the command line utilities required to complete this tutorial: [cfssl](https://github.com/cloudflare/cfssl), [cfssljson](https://github.com/cloudflare/cfssl), and [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl).
|
||||
In this lab you will install the command line utilities required to complete this tutorial: [step](https://github.com/smallstep/cli), and [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl).
|
||||
|
||||
|
||||
## Install CFSSL
|
||||
|
||||
The `cfssl` and `cfssljson` command line utilities will be used to provision a [PKI Infrastructure](https://en.wikipedia.org/wiki/Public_key_infrastructure) and generate TLS certificates.
|
||||
The `step` command line utility will be used to provision a [PKI Infrastructure](https://en.wikipedia.org/wiki/Public_key_infrastructure) and generate TLS certificates.
|
||||
|
||||
Download and install `cfssl` and `cfssljson`:
|
||||
Download and install `step`:
|
||||
|
||||
### OS X
|
||||
|
||||
```
|
||||
curl -o cfssl https://storage.googleapis.com/kubernetes-the-hard-way/cfssl/1.4.1/darwin/cfssl
|
||||
curl -o cfssljson https://storage.googleapis.com/kubernetes-the-hard-way/cfssl/1.4.1/darwin/cfssljson
|
||||
```
|
||||
For Intel chips:
|
||||
|
||||
```
|
||||
chmod +x cfssl cfssljson
|
||||
curl -L https://dl.step.sm/gh-release/cli/gh-release-header/v0.18.0/step_darwin_0.18.0_amd64.tar.gz | tar xz
|
||||
sudo mv step_0.18.0/bin/step /usr/local/bin/
|
||||
```
|
||||
|
||||
```
|
||||
sudo mv cfssl cfssljson /usr/local/bin/
|
||||
```
|
||||
|
||||
Some OS X users may experience problems using the pre-built binaries in which case [Homebrew](https://brew.sh) might be a better option:
|
||||
For Apple Silicon:
|
||||
|
||||
```
|
||||
brew install cfssl
|
||||
curl -L https://dl.step.sm/gh-release/cli/gh-release-header/v0.18.0/step_darwin_0.18.0_arm64.tar.gz | tar xz
|
||||
sudo mv step_0.18.0/bin/step /usr/local/bin/
|
||||
```
|
||||
|
||||
Or, if you'd like to use [Homebrew](https://brew.sh):
|
||||
|
||||
```
|
||||
brew install step
|
||||
```
|
||||
|
||||
### Linux
|
||||
|
||||
```
|
||||
wget -q --show-progress --https-only --timestamping \
|
||||
https://storage.googleapis.com/kubernetes-the-hard-way/cfssl/1.4.1/linux/cfssl \
|
||||
https://storage.googleapis.com/kubernetes-the-hard-way/cfssl/1.4.1/linux/cfssljson
|
||||
```
|
||||
|
||||
```
|
||||
chmod +x cfssl cfssljson
|
||||
```
|
||||
|
||||
```
|
||||
sudo mv cfssl cfssljson /usr/local/bin/
|
||||
curl -L https://dl.step.sm/gh-release/cli/gh-release-header/v0.18.0/step_linux_0.18.0_amd64.tar.gz
|
||||
sudo mv step_0.18.0/bin/step /usr/local/bin/
|
||||
```
|
||||
|
||||
### Verification
|
||||
|
||||
Verify `cfssl` and `cfssljson` version 1.4.1 or higher is installed:
|
||||
Verify `step` version 0.18.0 or higher is installed:
|
||||
|
||||
```
|
||||
cfssl version
|
||||
step version
|
||||
```
|
||||
|
||||
> output
|
||||
|
||||
```
|
||||
Version: 1.4.1
|
||||
Runtime: go1.12.12
|
||||
```
|
||||
|
||||
```
|
||||
cfssljson --version
|
||||
```
|
||||
```
|
||||
Version: 1.4.1
|
||||
Runtime: go1.12.12
|
||||
Smallstep CLI/0.18.0 (linux/amd64)
|
||||
Release Date: 2021-11-17 21:15 UTC
|
||||
```
|
||||
|
||||
## Install kubectl
|
||||
|
|
|
|||
|
|
@ -43,11 +43,11 @@ gcloud compute firewall-rules create kubernetes-the-hard-way-allow-internal \
|
|||
--source-ranges 10.240.0.0/24,10.200.0.0/16
|
||||
```
|
||||
|
||||
Create a firewall rule that allows external SSH, ICMP, and HTTPS:
|
||||
Create a firewall rule that allows external SSH, ICMP, HTTPS, and step-ca traffic:
|
||||
|
||||
```
|
||||
gcloud compute firewall-rules create kubernetes-the-hard-way-allow-external \
|
||||
--allow tcp:22,tcp:6443,icmp \
|
||||
--allow tcp:22,tcp:4443,tcp:6443,icmp \
|
||||
--network kubernetes-the-hard-way \
|
||||
--source-ranges 0.0.0.0/0
|
||||
```
|
||||
|
|
|
|||
|
|
@ -1,61 +1,265 @@
|
|||
# Provisioning a CA and Generating TLS Certificates
|
||||
# Provisioning a CA
|
||||
|
||||
In this lab you will provision a [PKI Infrastructure](https://en.wikipedia.org/wiki/Public_key_infrastructure) using CloudFlare's PKI toolkit, [cfssl](https://github.com/cloudflare/cfssl), then use it to bootstrap a Certificate Authority, and generate TLS certificates for the following components: etcd, kube-apiserver, kube-controller-manager, kube-scheduler, kubelet, and kube-proxy.
|
||||
In this lab you will provision a [PKI Infrastructure](https://en.wikipedia.org/wiki/Public_key_infrastructure) using Smallstep's CA server, [`step-ca`](https://github.com/smallstep/certificates), and generate TLS certificates for the following components: etcd, kube-apiserver, kube-controller-manager, kube-scheduler, kubelet, and kube-proxy.
|
||||
|
||||
## Certificate Authority
|
||||
|
||||
In this section you will provision a Certificate Authority that can be used to generate additional TLS certificates.
|
||||
In this section you will provision a `step-ca` Certificate Authority that can be used to generate additional TLS certificates. The CA will only run on `controller-0`. While it's possible to run a high-availability CA across multiple nodes, it's not necessary in a small-to-medium sized Kubernetes cluster. The CA service would have to be down for several days before having any negative impact on the cluster.
|
||||
|
||||
Generate the CA configuration file, certificate, and private key:
|
||||
Connect to `controller-0`:
|
||||
|
||||
```
|
||||
gcloud compute ssh controller-0
|
||||
```
|
||||
|
||||
Download the `step` client and `step-ca` server binaries, and the `jq` command:
|
||||
|
||||
```
|
||||
{
|
||||
wget -q --show-progress --https-only --timestamping \
|
||||
"https://dl.step.sm/gh-release/certificates/gh-release-header/v0.18.1/step-ca_linux_0.18.1_amd64.tar.gz" \
|
||||
"https://dl.step.sm/gh-release/cli/gh-release-header/v0.18.1/step_linux_0.18.1_amd64.tar.gz"
|
||||
sudo apt update
|
||||
sudo apt install -y jq
|
||||
}
|
||||
```
|
||||
|
||||
cat > ca-config.json <<EOF
|
||||
Install the binaries:
|
||||
|
||||
```
|
||||
{
|
||||
"signing": {
|
||||
"default": {
|
||||
"expiry": "8760h"
|
||||
tar -xvf step-ca_linux_0.18.1_amd64.tar.gz
|
||||
sudo mv step-ca_0.18.1/bin/step-ca /usr/local/bin/
|
||||
tar -xvf step_linux_0.18.1_amd64.tar.gz
|
||||
sudo mv step_0.18.1/bin/step /usr/local/bin/
|
||||
}
|
||||
```
|
||||
|
||||
Now create a `step` user and the paths for `step-ca`:
|
||||
|
||||
```
|
||||
sudo useradd --system --home /etc/step-ca --shell /bin/false step
|
||||
```
|
||||
|
||||
Create a CA configuration folder and generate passwords for the CA root key and the CA provisioner:
|
||||
|
||||
```
|
||||
{
|
||||
export STEPPATH=/etc/step-ca
|
||||
umask 077
|
||||
< /dev/urandom tr -dc A-Za-z0-9 | head -c40 | sudo tee $(step path)/password > /dev/null
|
||||
< /dev/urandom tr -dc A-Za-z0-9 | head -c40 > provisioner-password
|
||||
umask 002
|
||||
}
|
||||
```
|
||||
|
||||
Initialize your PKI:
|
||||
|
||||
```
|
||||
{
|
||||
INTERNAL_IP=$(curl -s -H "Metadata-Flavor: Google" \
|
||||
http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/ip)
|
||||
EXTERNAL_IP=$(curl -s -H "Metadata-Flavor: Google" http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip)
|
||||
INTERNAL_HOSTNAME=$(hostname -f)
|
||||
sudo -E step ca init --name="admin" \
|
||||
--dns="$INTERNAL_IP,$INTERNAL_HOSTNAME,$EXTERNAL_IP" \
|
||||
--address=":4443" --provisioner="kubernetes" \
|
||||
--password-file="$(step path)/password" \
|
||||
--provisioner-password-file="provisioner-password"
|
||||
}
|
||||
```
|
||||
|
||||
Add an X509 certificate template file:
|
||||
|
||||
```
|
||||
mkdir -p /etc/step-ca/templates/x509
|
||||
|
||||
# Server cert template.
|
||||
cat <<EOF > /etc/step-ca/templates/x509/kubernetes.tpl
|
||||
{
|
||||
"subject": {
|
||||
{{- if .Insecure.User.Organization }}
|
||||
"organization": {{ toJson .Insecure.User.Organization }},
|
||||
{{- end }}
|
||||
"commonName": {{ toJson .Subject.CommonName }},
|
||||
"organizationalUnit": {{ toJson .OrganizationalUnit }}
|
||||
},
|
||||
"profiles": {
|
||||
"kubernetes": {
|
||||
"usages": ["signing", "key encipherment", "server auth", "client auth"],
|
||||
"expiry": "8760h"
|
||||
}
|
||||
}
|
||||
}
|
||||
"sans": {{ toJson .SANs }},
|
||||
{{- if typeIs "*rsa.PublicKey" .Insecure.CR.PublicKey }}
|
||||
"keyUsage": ["keyEncipherment", "digitalSignature"],
|
||||
{{- else }}
|
||||
"keyUsage": ["digitalSignature"],
|
||||
{{- end }}
|
||||
"extKeyUsage": ["serverAuth", "clientAuth"]
|
||||
}
|
||||
EOF
|
||||
```
|
||||
|
||||
cat > ca-csr.json <<EOF
|
||||
Configure the CA provisioner to issue 90-day certificates:
|
||||
|
||||
```
|
||||
{
|
||||
"CN": "Kubernetes",
|
||||
"key": {
|
||||
"algo": "rsa",
|
||||
"size": 2048
|
||||
cat <<< $(jq '(.authority.provisioners[] | select(.name == "kubernetes")) += {
|
||||
"claims": {
|
||||
"maxTLSCertDuration": "2160h",
|
||||
"defaultTLSCertDuration": "2160h"
|
||||
},
|
||||
"names": [
|
||||
{
|
||||
"C": "US",
|
||||
"L": "Portland",
|
||||
"O": "Kubernetes",
|
||||
"OU": "CA",
|
||||
"ST": "Oregon"
|
||||
"options": {
|
||||
"x509": {
|
||||
"templateFile": "templates/x509/kubernetes.tpl",
|
||||
"templateData": {
|
||||
"OrganizationalUnit": "Kubernetes The Hard Way"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}' /etc/step-ca/config/ca.json) > /etc/step-ca/config/ca.json
|
||||
}
|
||||
```
|
||||
|
||||
Put the CA configuration into place, and add the CA to systemd:
|
||||
|
||||
```
|
||||
{
|
||||
sudo chown -R step:step /etc/step-ca
|
||||
cat <<EOF | sudo tee /etc/systemd/system/step-ca.service
|
||||
[Unit]
|
||||
Description=step-ca service
|
||||
Documentation=https://smallstep.com/docs/step-ca
|
||||
Documentation=https://smallstep.com/docs/step-ca/certificate-authority-server-production
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
StartLimitIntervalSec=30
|
||||
StartLimitBurst=3
|
||||
ConditionFileNotEmpty=/etc/step-ca/config/ca.json
|
||||
ConditionFileNotEmpty=/etc/step-ca/password
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=step
|
||||
Group=step
|
||||
Environment=STEPPATH=/etc/step-ca
|
||||
WorkingDirectory=/etc/step-ca
|
||||
ExecStart=/usr/local/bin/step-ca config/ca.json --password-file password
|
||||
ExecReload=/bin/kill --signal HUP $MAINPID
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
TimeoutStopSec=30
|
||||
StartLimitInterval=30
|
||||
StartLimitBurst=3
|
||||
|
||||
; Process capabilities & privileges
|
||||
AmbientCapabilities=CAP_NET_BIND_SERVICE
|
||||
CapabilityBoundingSet=CAP_NET_BIND_SERVICE
|
||||
SecureBits=keep-caps
|
||||
NoNewPrivileges=yes
|
||||
|
||||
; Sandboxing
|
||||
ProtectSystem=full
|
||||
ProtectHome=true
|
||||
RestrictNamespaces=true
|
||||
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
|
||||
PrivateTmp=true
|
||||
ProtectClock=true
|
||||
ProtectControlGroups=true
|
||||
ProtectKernelTunables=true
|
||||
ProtectKernelLogs=true
|
||||
ProtectKernelModules=true
|
||||
LockPersonality=true
|
||||
RestrictSUIDSGID=true
|
||||
RemoveIPC=true
|
||||
RestrictRealtime=true
|
||||
PrivateDevices=true
|
||||
SystemCallFilter=@system-service
|
||||
SystemCallArchitectures=native
|
||||
MemoryDenyWriteExecute=true
|
||||
ReadWriteDirectories=/etc/step-ca/db
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
EOF
|
||||
|
||||
cfssl gencert -initca ca-csr.json | cfssljson -bare ca
|
||||
|
||||
}
|
||||
```
|
||||
|
||||
Results:
|
||||
Save the root CA certificate:
|
||||
|
||||
```
|
||||
sudo cat /etc/step-ca/certs/root_ca.crt | tee ca.pem > /dev/null
|
||||
```
|
||||
|
||||
Finally, start the CA service:
|
||||
|
||||
```
|
||||
{
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now step-ca
|
||||
}
|
||||
```
|
||||
|
||||
## Verification
|
||||
|
||||
Check the CA health, then request and save the CA root certificate:
|
||||
|
||||
```
|
||||
sudo -u step -E step ca health
|
||||
```
|
||||
|
||||
Output:
|
||||
|
||||
```
|
||||
ok
|
||||
```
|
||||
|
||||
You can now sign out of `controller-0`.
|
||||
|
||||
# Generating certificates
|
||||
|
||||
## Bootstrapping with the CA
|
||||
|
||||
### Bootstrapping your local machine
|
||||
|
||||
Run the following on your local machine.
|
||||
|
||||
Download your CA's root certificate:
|
||||
|
||||
```
|
||||
gcloud compute scp controller-0:ca.pem controller-0:provisioner-password .
|
||||
```
|
||||
|
||||
Result:
|
||||
|
||||
```
|
||||
ca-key.pem
|
||||
ca.pem
|
||||
provisioner-password
|
||||
```
|
||||
|
||||
Now bootstrap with your CA:
|
||||
|
||||
```
|
||||
{
|
||||
CA_IP=$(gcloud compute instances describe controller-0 \
|
||||
--format='get(networkInterfaces[0].accessConfigs[0].natIP)')
|
||||
step ca bootstrap --ca-url "https://$CA_IP:4443/" --fingerprint $(step certificate fingerprint ca.pem)
|
||||
}
|
||||
```
|
||||
|
||||
Output:
|
||||
|
||||
```
|
||||
The root certificate has been saved in /home/carl/.step/authorities/XX.XXX.XXX.XXX/certs/root_ca.crt.
|
||||
The authority configuration has been saved in /home/carl/.step/authorities/XX.XXX.XXX.XXX/config/defaults.json.
|
||||
The profile configuration has been saved in /home/carl/.step/profiles/XX.XXX.XXX.XXX/config/defaults.json.
|
||||
```
|
||||
|
||||
Add your CA URL and fingerprint to the project metadata on GCP, so instances can bootstrap:
|
||||
|
||||
```
|
||||
gcloud compute project-info add-metadata --metadata="STEP_CA_URL=https://10.240.0.10:4443,STEP_CA_FINGERPRINT=$(step certificate fingerprint ca.pem)"
|
||||
```
|
||||
|
||||
Output:
|
||||
|
||||
```
|
||||
Updated [https://www.googleapis.com/compute/v1/projects/project-id-xxxxxx].
|
||||
```
|
||||
|
||||
## Client and Server Certificates
|
||||
|
|
@ -64,37 +268,15 @@ In this section you will generate client and server certificates for each Kubern
|
|||
|
||||
### The Admin Client Certificate
|
||||
|
||||
Generate the `admin` client certificate and private key:
|
||||
On your local machine, generate the `admin` client certificate and private key:
|
||||
|
||||
```
|
||||
{
|
||||
|
||||
cat > admin-csr.json <<EOF
|
||||
{
|
||||
"CN": "admin",
|
||||
"key": {
|
||||
"algo": "rsa",
|
||||
"size": 2048
|
||||
},
|
||||
"names": [
|
||||
{
|
||||
"C": "US",
|
||||
"L": "Portland",
|
||||
"O": "system:masters",
|
||||
"OU": "Kubernetes The Hard Way",
|
||||
"ST": "Oregon"
|
||||
}
|
||||
]
|
||||
}
|
||||
EOF
|
||||
|
||||
cfssl gencert \
|
||||
-ca=ca.pem \
|
||||
-ca-key=ca-key.pem \
|
||||
-config=ca-config.json \
|
||||
-profile=kubernetes \
|
||||
admin-csr.json | cfssljson -bare admin
|
||||
|
||||
step ca certificate admin admin.pem admin-key.pem \
|
||||
--provisioner="kubernetes" \
|
||||
--provisioner-password-file="provisioner-password" \
|
||||
--set "Organization=system:masters" \
|
||||
--kty RSA
|
||||
}
|
||||
```
|
||||
|
||||
|
|
@ -113,24 +295,6 @@ Generate a certificate and private key for each Kubernetes worker node:
|
|||
|
||||
```
|
||||
for instance in worker-0 worker-1 worker-2; do
|
||||
cat > ${instance}-csr.json <<EOF
|
||||
{
|
||||
"CN": "system:node:${instance}",
|
||||
"key": {
|
||||
"algo": "rsa",
|
||||
"size": 2048
|
||||
},
|
||||
"names": [
|
||||
{
|
||||
"C": "US",
|
||||
"L": "Portland",
|
||||
"O": "system:nodes",
|
||||
"OU": "Kubernetes The Hard Way",
|
||||
"ST": "Oregon"
|
||||
}
|
||||
]
|
||||
}
|
||||
EOF
|
||||
|
||||
EXTERNAL_IP=$(gcloud compute instances describe ${instance} \
|
||||
--format 'value(networkInterfaces[0].accessConfigs[0].natIP)')
|
||||
|
|
@ -138,13 +302,13 @@ EXTERNAL_IP=$(gcloud compute instances describe ${instance} \
|
|||
INTERNAL_IP=$(gcloud compute instances describe ${instance} \
|
||||
--format 'value(networkInterfaces[0].networkIP)')
|
||||
|
||||
cfssl gencert \
|
||||
-ca=ca.pem \
|
||||
-ca-key=ca-key.pem \
|
||||
-config=ca-config.json \
|
||||
-hostname=${instance},${EXTERNAL_IP},${INTERNAL_IP} \
|
||||
-profile=kubernetes \
|
||||
${instance}-csr.json | cfssljson -bare ${instance}
|
||||
step ca certificate "system:node:${instance}" ${instance}.pem ${instance}-key.pem \
|
||||
--san "${instance}" \
|
||||
--san "${EXTERNAL_IP}" \
|
||||
--san "${INTERNAL_IP}" \
|
||||
--set "Organization=system:nodes" \
|
||||
--provisioner "kubernetes" \
|
||||
--provisioner-password-file "provisioner-password"
|
||||
done
|
||||
```
|
||||
|
||||
|
|
@ -161,37 +325,25 @@ worker-2.pem
|
|||
|
||||
### The Controller Manager Client Certificate
|
||||
|
||||
Generate the `kube-controller-manager` client certificate and private key:
|
||||
Generate the `kube-controller-manager`, `kube-proxy`, and `kube-scheduler` client certificates and private keys:
|
||||
|
||||
```
|
||||
{
|
||||
|
||||
cat > kube-controller-manager-csr.json <<EOF
|
||||
{
|
||||
"CN": "system:kube-controller-manager",
|
||||
"key": {
|
||||
"algo": "rsa",
|
||||
"size": 2048
|
||||
},
|
||||
"names": [
|
||||
{
|
||||
"C": "US",
|
||||
"L": "Portland",
|
||||
"O": "system:kube-controller-manager",
|
||||
"OU": "Kubernetes The Hard Way",
|
||||
"ST": "Oregon"
|
||||
}
|
||||
]
|
||||
}
|
||||
EOF
|
||||
|
||||
cfssl gencert \
|
||||
-ca=ca.pem \
|
||||
-ca-key=ca-key.pem \
|
||||
-config=ca-config.json \
|
||||
-profile=kubernetes \
|
||||
kube-controller-manager-csr.json | cfssljson -bare kube-controller-manager
|
||||
|
||||
step ca certificate "system:kube-controller-manager" kube-controller-manager.pem kube-controller-manager-key.pem \
|
||||
--kty RSA \
|
||||
--set "Organization=system:kube-controller-manager" \
|
||||
--provisioner "kubernetes" \
|
||||
--provisioner-password-file "provisioner-password"
|
||||
step ca certificate "system:kube-proxy" kube-proxy.pem kube-proxy-key.pem \
|
||||
--kty RSA \
|
||||
--set "Organization=system:node-proxier" \
|
||||
--provisioner "kubernetes" \
|
||||
--provisioner-password-file "provisioner-password"
|
||||
step ca certificate "system:kube-scheduler" kube-scheduler.pem kube-scheduler-key.pem \
|
||||
--kty RSA \
|
||||
--set "Organization=system:kube-scheduler" \
|
||||
--provisioner "kubernetes" \
|
||||
--provisioner-password-file "provisioner-password"
|
||||
}
|
||||
```
|
||||
|
||||
|
|
@ -200,96 +352,12 @@ Results:
|
|||
```
|
||||
kube-controller-manager-key.pem
|
||||
kube-controller-manager.pem
|
||||
```
|
||||
|
||||
|
||||
### The Kube Proxy Client Certificate
|
||||
|
||||
Generate the `kube-proxy` client certificate and private key:
|
||||
|
||||
```
|
||||
{
|
||||
|
||||
cat > kube-proxy-csr.json <<EOF
|
||||
{
|
||||
"CN": "system:kube-proxy",
|
||||
"key": {
|
||||
"algo": "rsa",
|
||||
"size": 2048
|
||||
},
|
||||
"names": [
|
||||
{
|
||||
"C": "US",
|
||||
"L": "Portland",
|
||||
"O": "system:node-proxier",
|
||||
"OU": "Kubernetes The Hard Way",
|
||||
"ST": "Oregon"
|
||||
}
|
||||
]
|
||||
}
|
||||
EOF
|
||||
|
||||
cfssl gencert \
|
||||
-ca=ca.pem \
|
||||
-ca-key=ca-key.pem \
|
||||
-config=ca-config.json \
|
||||
-profile=kubernetes \
|
||||
kube-proxy-csr.json | cfssljson -bare kube-proxy
|
||||
|
||||
}
|
||||
```
|
||||
|
||||
Results:
|
||||
|
||||
```
|
||||
kube-proxy-key.pem
|
||||
kube-proxy.pem
|
||||
```
|
||||
|
||||
### The Scheduler Client Certificate
|
||||
|
||||
Generate the `kube-scheduler` client certificate and private key:
|
||||
|
||||
```
|
||||
{
|
||||
|
||||
cat > kube-scheduler-csr.json <<EOF
|
||||
{
|
||||
"CN": "system:kube-scheduler",
|
||||
"key": {
|
||||
"algo": "rsa",
|
||||
"size": 2048
|
||||
},
|
||||
"names": [
|
||||
{
|
||||
"C": "US",
|
||||
"L": "Portland",
|
||||
"O": "system:kube-scheduler",
|
||||
"OU": "Kubernetes The Hard Way",
|
||||
"ST": "Oregon"
|
||||
}
|
||||
]
|
||||
}
|
||||
EOF
|
||||
|
||||
cfssl gencert \
|
||||
-ca=ca.pem \
|
||||
-ca-key=ca-key.pem \
|
||||
-config=ca-config.json \
|
||||
-profile=kubernetes \
|
||||
kube-scheduler-csr.json | cfssljson -bare kube-scheduler
|
||||
|
||||
}
|
||||
```
|
||||
|
||||
Results:
|
||||
|
||||
```
|
||||
kube-scheduler-key.pem
|
||||
kube-scheduler.pem
|
||||
```
|
||||
|
||||
|
||||
### The Kubernetes API Server Certificate
|
||||
|
||||
The `kubernetes-the-hard-way` static IP address will be included in the list of subject alternative names for the Kubernetes API Server certificate. This will ensure the certificate can be validated by remote clients.
|
||||
|
|
@ -298,40 +366,25 @@ Generate the Kubernetes API Server certificate and private key:
|
|||
|
||||
```
|
||||
{
|
||||
|
||||
KUBERNETES_PUBLIC_ADDRESS=$(gcloud compute addresses describe kubernetes-the-hard-way \
|
||||
--region $(gcloud config get-value compute/region) \
|
||||
--format 'value(address)')
|
||||
|
||||
KUBERNETES_HOSTNAMES=kubernetes,kubernetes.default,kubernetes.default.svc,kubernetes.default.svc.cluster,kubernetes.svc.cluster.local
|
||||
|
||||
cat > kubernetes-csr.json <<EOF
|
||||
{
|
||||
"CN": "kubernetes",
|
||||
"key": {
|
||||
"algo": "rsa",
|
||||
"size": 2048
|
||||
},
|
||||
"names": [
|
||||
{
|
||||
"C": "US",
|
||||
"L": "Portland",
|
||||
"O": "Kubernetes",
|
||||
"OU": "Kubernetes The Hard Way",
|
||||
"ST": "Oregon"
|
||||
}
|
||||
]
|
||||
}
|
||||
EOF
|
||||
|
||||
cfssl gencert \
|
||||
-ca=ca.pem \
|
||||
-ca-key=ca-key.pem \
|
||||
-config=ca-config.json \
|
||||
-hostname=10.32.0.1,10.240.0.10,10.240.0.11,10.240.0.12,${KUBERNETES_PUBLIC_ADDRESS},127.0.0.1,${KUBERNETES_HOSTNAMES} \
|
||||
-profile=kubernetes \
|
||||
kubernetes-csr.json | cfssljson -bare kubernetes
|
||||
|
||||
step ca certificate "kubernetes" kubernetes.pem kubernetes-key.pem \
|
||||
--kty RSA \
|
||||
--san kubernetes \
|
||||
--san kubernetes.default \
|
||||
--san kubernetes.default.svc \
|
||||
--san kubernetes.default.svc.cluster \
|
||||
--san kubernetes.default.svc.cluster.local \
|
||||
--san 10.32.0.1 \
|
||||
--san 10.240.0.10 \
|
||||
--san 10.240.0.11 \
|
||||
--san 10.240.0.12 \
|
||||
--san ${KUBERNETES_PUBLIC_ADDRESS} \
|
||||
--san 127.0.0.1 \
|
||||
--set "Organization=Kubernetes" \
|
||||
--provisioner "kubernetes" \
|
||||
--provisioner-password-file "provisioner-password"
|
||||
}
|
||||
```
|
||||
|
||||
|
|
@ -352,33 +405,11 @@ Generate the `service-account` certificate and private key:
|
|||
|
||||
```
|
||||
{
|
||||
|
||||
cat > service-account-csr.json <<EOF
|
||||
{
|
||||
"CN": "service-accounts",
|
||||
"key": {
|
||||
"algo": "rsa",
|
||||
"size": 2048
|
||||
},
|
||||
"names": [
|
||||
{
|
||||
"C": "US",
|
||||
"L": "Portland",
|
||||
"O": "Kubernetes",
|
||||
"OU": "Kubernetes The Hard Way",
|
||||
"ST": "Oregon"
|
||||
}
|
||||
]
|
||||
}
|
||||
EOF
|
||||
|
||||
cfssl gencert \
|
||||
-ca=ca.pem \
|
||||
-ca-key=ca-key.pem \
|
||||
-config=ca-config.json \
|
||||
-profile=kubernetes \
|
||||
service-account-csr.json | cfssljson -bare service-account
|
||||
|
||||
step ca certificate "service-accounts" service-account.pem service-account-key.pem \
|
||||
--kty RSA \
|
||||
--set "Organization=Kubernetes" \
|
||||
--provisioner "kubernetes" \
|
||||
--provisioner-password-file "provisioner-password"
|
||||
}
|
||||
```
|
||||
|
||||
|
|
@ -396,7 +427,8 @@ Copy the appropriate certificates and private keys to each worker instance:
|
|||
|
||||
```
|
||||
for instance in worker-0 worker-1 worker-2; do
|
||||
gcloud compute scp ca.pem ${instance}-key.pem ${instance}.pem ${instance}:~/
|
||||
gcloud compute scp ca.pem ${instance}-key.pem ${instance}.pem \
|
||||
kube-proxy-key.pem kube-proxy.pem ${instance}:~/
|
||||
done
|
||||
```
|
||||
|
||||
|
|
@ -404,8 +436,10 @@ Copy the appropriate certificates and private keys to each controller instance:
|
|||
|
||||
```
|
||||
for instance in controller-0 controller-1 controller-2; do
|
||||
gcloud compute scp ca.pem ca-key.pem kubernetes-key.pem kubernetes.pem \
|
||||
service-account-key.pem service-account.pem ${instance}:~/
|
||||
gcloud compute scp ca.pem kubernetes-key.pem kubernetes.pem \
|
||||
service-account-key.pem service-account.pem \
|
||||
kube-controller-manager-key.pem kube-controller-manager.pem \
|
||||
kube-scheduler-key.pem kube-scheduler.pem ${instance}:~/
|
||||
done
|
||||
```
|
||||
|
||||
|
|
|
|||
|
|
@ -49,8 +49,11 @@ Install the Kubernetes binaries:
|
|||
{
|
||||
sudo mkdir -p /var/lib/kubernetes/
|
||||
|
||||
sudo mv ca.pem ca-key.pem kubernetes-key.pem kubernetes.pem \
|
||||
sudo mv ca.pem kubernetes-key.pem kubernetes.pem \
|
||||
service-account-key.pem service-account.pem \
|
||||
kube-proxy.pem kube-proxy-key.pem \
|
||||
kube-controller-manager.pem kube-controller-manager-key.pem \
|
||||
kube-scheduler.pem kube-scheduler-key.pem \
|
||||
encryption-config.yaml /var/lib/kubernetes/
|
||||
}
|
||||
```
|
||||
|
|
@ -142,7 +145,6 @@ ExecStart=/usr/local/bin/kube-controller-manager \\
|
|||
--cluster-cidr=10.200.0.0/16 \\
|
||||
--cluster-name=kubernetes \\
|
||||
--cluster-signing-cert-file=/var/lib/kubernetes/ca.pem \\
|
||||
--cluster-signing-key-file=/var/lib/kubernetes/ca-key.pem \\
|
||||
--kubeconfig=/var/lib/kubernetes/kube-controller-manager.kubeconfig \\
|
||||
--leader-elect=true \\
|
||||
--root-ca-file=/var/lib/kubernetes/ca.pem \\
|
||||
|
|
|
|||
|
|
@ -244,6 +244,7 @@ EOF
|
|||
|
||||
```
|
||||
sudo mv kube-proxy.kubeconfig /var/lib/kube-proxy/kubeconfig
|
||||
sudo mv kube-proxy.pem kube-proxy-key.pem /var/lib/kube-proxy
|
||||
```
|
||||
|
||||
Create the `kube-proxy-config.yaml` configuration file:
|
||||
|
|
|
|||
|
|
@ -0,0 +1,279 @@
|
|||
# Configuring Certificate Renewal
|
||||
|
||||
## Prerequisites
|
||||
|
||||
The commands in this section must be run on every instance: `controller-0`, `controller-1`, `controller-2`, `worker-0`, `worker-1`, and `worker-2`. Login to each instance using the `gcloud` command. Example:
|
||||
|
||||
```
|
||||
gcloud compute ssh controller-0
|
||||
```
|
||||
|
||||
## Download certificate management tools
|
||||
|
||||
Run each command on every node.
|
||||
|
||||
Download the `step` CLI binary:
|
||||
|
||||
```
|
||||
wget -q --show-progress --https-only --timestamping \
|
||||
"https://dl.step.sm/gh-release/cli/gh-release-header/v0.18.1/step_linux_0.18.1_amd64.tar.gz"
|
||||
```
|
||||
|
||||
Install the binary:
|
||||
|
||||
```
|
||||
tar -xvf step_linux_0.18.1_amd64.tar.gz
|
||||
sudo mv step_0.18.1/bin/step /usr/local/bin/
|
||||
```
|
||||
|
||||
## Bootstrap with the CA
|
||||
|
||||
Configure the host to trust your Certificate Authority:
|
||||
|
||||
```
|
||||
{
|
||||
STEP_CA_URL=$(gcloud compute project-info describe --format='get(commonInstanceMetadata.items.STEP_CA_URL)')
|
||||
STEP_CA_FINGERPRINT=$(gcloud compute project-info describe --format='get(commonInstanceMetadata.items.STEP_CA_FINGERPRINT)')
|
||||
sudo step ca bootstrap \
|
||||
--ca-url "${STEP_CA_URL}" \
|
||||
--fingerprint "${STEP_CA_FINGERPRINT}"
|
||||
}
|
||||
```
|
||||
|
||||
Output:
|
||||
|
||||
```
|
||||
The root certificate has been saved in /root/.step/certs/root_ca.crt.
|
||||
The authority configuration has been saved in /root/.step/config/defaults.json.
|
||||
```
|
||||
|
||||
## Set up the certificate renewal timer
|
||||
|
||||
We'll use a systemd timer to renew certificates when they are 2/3rds of the way through their validity period.
|
||||
|
||||
Install the systemd certificate renewal service and timer.
|
||||
|
||||
```
|
||||
cat << EOF | sudo tee /etc/systemd/system/cert-renewer@.service
|
||||
[Unit]
|
||||
Description=Certificate renewer for %I
|
||||
After=network-online.target
|
||||
Documentation=https://smallstep.com/docs/step-ca/certificate-authority-server-production
|
||||
StartLimitIntervalSec=0
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
User=root
|
||||
|
||||
Environment=STEPPATH=/etc/step-ca \\
|
||||
CERT_LOCATION=/etc/step/certs/%i.crt \\
|
||||
KEY_LOCATION=/etc/step/certs/%i.key
|
||||
|
||||
; ExecCondition checks if the certificate is ready for renewal,
|
||||
; based on the exit status of the command.
|
||||
; (In systemd <242, you can use ExecStartPre= here.)
|
||||
ExecCondition=/usr/local/bin/step certificate needs-renewal \${CERT_LOCATION}
|
||||
|
||||
; ExecStart renews the certificate, if ExecStartPre was successful.
|
||||
ExecStart=/usr/local/bin/step ca renew --force \${CERT_LOCATION} \${KEY_LOCATION}
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
EOF
|
||||
```
|
||||
|
||||
Install the timer:
|
||||
|
||||
```
|
||||
cat << EOF | sudo tee /etc/systemd/system/cert-renewer@.timer
|
||||
[Unit]
|
||||
Description=Certificate renewal timer for %I
|
||||
Documentation=https://smallstep.com/docs/step-ca/certificate-authority-server-production
|
||||
|
||||
[Timer]
|
||||
Persistent=true
|
||||
|
||||
; Run the timer unit every 5 minutes.
|
||||
OnCalendar=*:1/5
|
||||
|
||||
; Always run the timer on time.
|
||||
AccuracySec=1us
|
||||
|
||||
; Add jitter to prevent a "thundering hurd" of simultaneous certificate renewals.
|
||||
RandomizedDelaySec=5m
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
EOF
|
||||
```
|
||||
|
||||
# Controller Certificate Renewal
|
||||
|
||||
## Prerequisites
|
||||
|
||||
The commands in this section must be run on every controller: `controller-0`, `controller-1`, `controller-2`. Login to each instance using the `gcloud` command. Example:
|
||||
|
||||
```
|
||||
gcloud compute ssh controller-0
|
||||
```
|
||||
|
||||
## Configure certificate renewal for etcd
|
||||
|
||||
Create and start a certificate renewal timer for etcd:
|
||||
|
||||
```
|
||||
sudo mkdir /etc/systemd/system/cert-renewer@etcd.service.d
|
||||
cat <<EOF | sudo tee /etc/systemd/system/cert-renewer@etcd.service.d/override.conf
|
||||
[Service]
|
||||
Environment=STEPPATH=/root/.step \\
|
||||
CERT_LOCATION=/etc/etcd/kubernetes.pem \\
|
||||
KEY_LOCATION=/etc/etcd/kubernetes-key.pem
|
||||
EOF
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now cert-renewer@etcd.timer
|
||||
```
|
||||
|
||||
## Configure certificate renewal for `kube-controller-manager`
|
||||
|
||||
Create and start a certificate renewal timer for `kube-controller-manager`. This one will use `kubectl` to embed the renewed certificate and key into the kubeconfig file before restarting the controller manager. Run:
|
||||
|
||||
```
|
||||
sudo mkdir /etc/systemd/system/cert-renewer@kube-controller-manager.service.d
|
||||
cat <<EOF | sudo tee /etc/systemd/system/cert-renewer@kube-controller-manager.service.d/override.conf
|
||||
[Service]
|
||||
Environment=STEPPATH=/root/.step \\
|
||||
CERT_LOCATION=/var/lib/kubernetes/kube-controller-manager.pem \\
|
||||
KEY_LOCATION=/var/lib/kubernetes/kube-controller-manager-key.pem
|
||||
|
||||
ExecStartPost=kubectl config set-credentials system:kube-controller-manager \\
|
||||
--client-certificate=\${CERT_LOCATION} \\
|
||||
--client-key=\${KEY_LOCATION} \\
|
||||
--embed-certs=true \\
|
||||
--kubeconfig=/var/lib/kubernetes/kube-controller-manager.kubeconfig
|
||||
|
||||
ExecStartPost=systemctl restart kube-controller-manager.service
|
||||
EOF
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now cert-renewer@kube-controller-manager.timer
|
||||
```
|
||||
|
||||
## Configure certificate renewal for kube-scheduler
|
||||
|
||||
Create and start a certificate renewal timer for `kube-scheduler`:
|
||||
|
||||
```
|
||||
sudo mkdir /etc/systemd/system/cert-renewer@kube-scheduler.service.d
|
||||
cat <<EOF | sudo tee /etc/systemd/system/cert-renewer@kube-scheduler.service.d/override.conf
|
||||
[Service]
|
||||
Environment=STEPPATH=/root/.step \\
|
||||
CERT_LOCATION=/var/lib/kubernetes/kube-scheduler.pem \\
|
||||
KEY_LOCATION=/var/lib/kubernetes/kube-scheduler-key.pem
|
||||
|
||||
ExecStartPost=kubectl config set-credentials system:kube-scheduler \\
|
||||
--client-certificate=\${CERT_LOCATION} \\
|
||||
--client-key=\${KEY_LOCATION} \\
|
||||
--embed-certs=true \\
|
||||
--kubeconfig=/var/lib/kubernetes/kube-scheduler.kubeconfig
|
||||
|
||||
ExecStartPost=systemctl restart kube-scheduler.service
|
||||
EOF
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now cert-renewer@kube-scheduler.timer
|
||||
```
|
||||
|
||||
|
||||
## Configure certificate renewal for kube-apiserver
|
||||
|
||||
Create and start a certificate renewal timer for `kube-apiserver`:
|
||||
|
||||
```
|
||||
sudo mkdir /etc/systemd/system/cert-renewer@kube-apiserver.service.d
|
||||
cat <<EOF | sudo tee /etc/systemd/system/cert-renewer@kube-apiserver.service.d/override.conf
|
||||
[Service]
|
||||
Environment=STEPPATH=/root/.step \\
|
||||
CERT_LOCATION=/var/lib/kubernetes/kubernetes.pem \\
|
||||
KEY_LOCATION=/var/lib/kubernetes/kubernetes-key.pem
|
||||
|
||||
ExecStartPost=systemctl restart kube-apiserver.service
|
||||
EOF
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now cert-renewer@kube-apiserver.timer
|
||||
```
|
||||
|
||||
|
||||
## Configure service account certificate renewal timer
|
||||
|
||||
The service account certificate and key is used by the API server, so we will need to restart it when the certificate file is updated:
|
||||
|
||||
```
|
||||
sudo mkdir /etc/systemd/system/cert-renewer@kube-service-account.service.d
|
||||
cat <<EOF | sudo tee /etc/systemd/system/cert-renewer@kube-service-account.service.d/override.conf
|
||||
[Service]
|
||||
Environment=STEPPATH=/root/.step \\
|
||||
CERT_LOCATION=/var/lib/kubernetes/service-account.pem \\
|
||||
KEY_LOCATION=/var/lib/kubernetes/service-account-key.pem
|
||||
|
||||
; Restart services that use the service account certificate or key
|
||||
ExecStartPost=systemctl restart kube-apiserver.service
|
||||
EOF
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now cert-renewer@kube-service-account.timer
|
||||
```
|
||||
|
||||
> Remember to run the above commands on each controller node: `controller-0`, `controller-1`, and `controller-2`.
|
||||
|
||||
# Worker Certificate Renewal
|
||||
|
||||
## Prerequisites
|
||||
|
||||
The commands in this section must be run on every worker: `worker-0`, `worker-1`, and `worker-2`. Login to each instance using the `gcloud` command. Example:
|
||||
|
||||
```
|
||||
gcloud compute ssh worker-0
|
||||
```
|
||||
|
||||
## Configure Certificate Renewal for `kubelet.service`
|
||||
|
||||
Install the a renewal service that will restart `kubelet.service` when the certificate is renewed:
|
||||
|
||||
```
|
||||
sudo mkdir /etc/systemd/system/cert-renewer@kubelet.service.d
|
||||
cat <<EOF | sudo tee /etc/systemd/system/cert-renewer@kubelet.service.d/override.conf
|
||||
[Service]
|
||||
Environment=STEPPATH=/root/.step \\
|
||||
CERT_LOCATION=/var/lib/kubelet/${HOSTNAME}.pem \\
|
||||
KEY_LOCATION=/var/lib/kubelet/${HOSTNAME}-key.pem
|
||||
|
||||
; Restart services that use the service account certificate or key
|
||||
ExecStartPost=systemctl restart kubelet.service
|
||||
EOF
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now cert-renewer@kubelet.timer
|
||||
```
|
||||
|
||||
## Configure Certificate Renewal for `kube-proxy.service`
|
||||
|
||||
Install a renewal service that will rebuild the kubeconfig file and restart kube-proxy when the certificate is renewed:
|
||||
|
||||
```
|
||||
sudo mkdir /etc/systemd/system/cert-renewer@kube-proxy.service.d
|
||||
cat <<EOF | sudo tee /etc/systemd/system/cert-renewer@kube-proxy.service.d/override.conf
|
||||
[Service]
|
||||
Environment=STEPPATH=/root/.step \\
|
||||
CERT_LOCATION=/var/lib/kube-proxy/kube-proxy.pem \\
|
||||
KEY_LOCATION=/var/lib/kube-proxy/kube-proxy.pem
|
||||
|
||||
ExecStartPost=kubectl config set-credentials system:kube-proxy \\
|
||||
--client-certificate=\${CERT_LOCATION} \\
|
||||
--client-key=\${KEY_LOCATION} \\
|
||||
--embed-certs=true \\
|
||||
--kubeconfig=/var/lib/kube-proxy/kubeconfig
|
||||
|
||||
ExecStartPost=systemctl restart kube-proxy.service
|
||||
EOF
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now cert-renewer@kube-proxy.timer
|
||||
```
|
||||
|
||||
|
||||
> Remember to run the above commands on each controller node: `worker-0`, `worker-1`, and `worker-2`.
|
||||
Loading…
Reference in New Issue