mirrors
/
kubernetes-the-hard-way
mirror of https://github.com/kelseyhightower/kubernetes-the-hard-way
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Merge 78a9c6863a into b974042d95

pull/381/merge
Luis San Martin 2018-08-26 16:17:03 +00:00 committed by GitHub
parent b974042d95 78a9c6863a
commit 3715036ec6
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
7 changed files with 163 additions and 205 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

41
docs/04-certificate-authority.md
View File

@ -8,9 +8,9 @@ In this section you will provision a Certificate Authority that can be used to g
Generate the CA configuration file, certificate, and private key: Generate the CA configuration file, certificate, and private key:
``` Have in mind all of the following commands must be done on your terminal.
{
```
cat > ca-config.json <<EOF cat > ca-config.json <<EOF
{ {
"signing": { "signing": {
@ -26,7 +26,9 @@ cat > ca-config.json <<EOF
} }
} }
EOF EOF
```
```
cat > ca-csr.json <<EOF cat > ca-csr.json <<EOF
{ {
"CN": "Kubernetes", "CN": "Kubernetes",
@ -45,13 +47,13 @@ cat > ca-csr.json <<EOF
] ]
} }
EOF EOF
cfssl gencert -initca ca-csr.json | cfssljson -bare ca
}
``` ```
Results: ```
cfssl gencert -initca ca-csr.json | cfssljson -bare ca
```
It should create two files:
``` ```
ca-key.pem ca-key.pem
@ -67,8 +69,6 @@ In this section you will generate client and server certificates for each Kubern
Generate the `admin` client certificate and private key: Generate the `admin` client certificate and private key:
``` ```
{
cat > admin-csr.json <<EOF cat > admin-csr.json <<EOF
{ {
"CN": "admin", "CN": "admin",
@ -87,18 +87,18 @@ cat > admin-csr.json <<EOF
] ]
} }
EOF EOF
```
```
cfssl gencert \ cfssl gencert \
-ca=ca.pem \ -ca=ca.pem \
-ca-key=ca-key.pem \ -ca-key=ca-key.pem \
-config=ca-config.json \ -config=ca-config.json \
-profile=kubernetes \ -profile=kubernetes \
admin-csr.json | cfssljson -bare admin admin-csr.json | cfssljson -bare admin
}
``` ```
Results: It should generate two additional files:
``` ```
admin-key.pem admin-key.pem
@ -164,8 +164,6 @@ worker-2.pem
Generate the `kube-controller-manager` client certificate and private key: Generate the `kube-controller-manager` client certificate and private key:
``` ```
{
cat > kube-controller-manager-csr.json <<EOF cat > kube-controller-manager-csr.json <<EOF
{ {
"CN": "system:kube-controller-manager", "CN": "system:kube-controller-manager",
@ -191,8 +189,6 @@ cfssl gencert \
-config=ca-config.json \ -config=ca-config.json \
-profile=kubernetes \ -profile=kubernetes \
kube-controller-manager-csr.json | cfssljson -bare kube-controller-manager kube-controller-manager-csr.json | cfssljson -bare kube-controller-manager
}
``` ```
Results: Results:
@ -208,8 +204,6 @@ kube-controller-manager.pem
Generate the `kube-proxy` client certificate and private key: Generate the `kube-proxy` client certificate and private key:
``` ```
{
cat > kube-proxy-csr.json <<EOF cat > kube-proxy-csr.json <<EOF
{ {
"CN": "system:kube-proxy", "CN": "system:kube-proxy",
@ -236,7 +230,6 @@ cfssl gencert \
-profile=kubernetes \ -profile=kubernetes \
kube-proxy-csr.json | cfssljson -bare kube-proxy kube-proxy-csr.json | cfssljson -bare kube-proxy
}
``` ```
Results: Results:
@ -251,8 +244,6 @@ kube-proxy.pem
Generate the `kube-scheduler` client certificate and private key: Generate the `kube-scheduler` client certificate and private key:
``` ```
{
cat > kube-scheduler-csr.json <<EOF cat > kube-scheduler-csr.json <<EOF
{ {
"CN": "system:kube-scheduler", "CN": "system:kube-scheduler",
@ -279,7 +270,6 @@ cfssl gencert \
-profile=kubernetes \ -profile=kubernetes \
kube-scheduler-csr.json | cfssljson -bare kube-scheduler kube-scheduler-csr.json | cfssljson -bare kube-scheduler
}
``` ```
Results: Results:
@ -297,8 +287,6 @@ The `kubernetes-the-hard-way` static IP address will be included in the list of
Generate the Kubernetes API Server certificate and private key: Generate the Kubernetes API Server certificate and private key:
``` ```
{
KUBERNETES_PUBLIC_ADDRESS=$(gcloud compute addresses describe kubernetes-the-hard-way \ KUBERNETES_PUBLIC_ADDRESS=$(gcloud compute addresses describe kubernetes-the-hard-way \
--region $(gcloud config get-value compute/region) \ --region $(gcloud config get-value compute/region) \
--format 'value(address)') --format 'value(address)')
@ -330,7 +318,6 @@ cfssl gencert \
-profile=kubernetes \ -profile=kubernetes \
kubernetes-csr.json | cfssljson -bare kubernetes kubernetes-csr.json | cfssljson -bare kubernetes
}
``` ```
Results: Results:
@ -347,8 +334,6 @@ The Kubernetes Controller Manager leverages a key pair to generate and sign serv
Generate the `service-account` certificate and private key: Generate the `service-account` certificate and private key:
``` ```
{
cat > service-account-csr.json <<EOF cat > service-account-csr.json <<EOF
{ {
"CN": "service-accounts", "CN": "service-accounts",
@ -374,8 +359,6 @@ cfssl gencert \
-config=ca-config.json \ -config=ca-config.json \
-profile=kubernetes \ -profile=kubernetes \
service-account-csr.json | cfssljson -bare service-account service-account-csr.json | cfssljson -bare service-account
}
``` ```
Results: Results:

128
docs/05-kubernetes-configuration-files.md
View File

@ -60,26 +60,24 @@ worker-2.kubeconfig
Generate a kubeconfig file for the `kube-proxy` service: Generate a kubeconfig file for the `kube-proxy` service:
``` ```
{ kubectl config set-cluster kubernetes-the-hard-way \
kubectl config set-cluster kubernetes-the-hard-way \ --certificate-authority=ca.pem \
--certificate-authority=ca.pem \ --embed-certs=true \
--embed-certs=true \ --server=https://${KUBERNETES_PUBLIC_ADDRESS}:6443 \
--server=https://${KUBERNETES_PUBLIC_ADDRESS}:6443 \ --kubeconfig=kube-proxy.kubeconfig
--kubeconfig=kube-proxy.kubeconfig
kubectl config set-credentials system:kube-proxy \ kubectl config set-credentials system:kube-proxy \
--client-certificate=kube-proxy.pem \ --client-certificate=kube-proxy.pem \
--client-key=kube-proxy-key.pem \ --client-key=kube-proxy-key.pem \
--embed-certs=true \ --embed-certs=true \
--kubeconfig=kube-proxy.kubeconfig --kubeconfig=kube-proxy.kubeconfig
kubectl config set-context default \ kubectl config set-context default \
--cluster=kubernetes-the-hard-way \ --cluster=kubernetes-the-hard-way \
--user=system:kube-proxy \ --user=system:kube-proxy \
--kubeconfig=kube-proxy.kubeconfig --kubeconfig=kube-proxy.kubeconfig
kubectl config use-context default --kubeconfig=kube-proxy.kubeconfig kubectl config use-context default --kubeconfig=kube-proxy.kubeconfig
}
``` ```
Results: Results:
@ -93,26 +91,24 @@ kube-proxy.kubeconfig
Generate a kubeconfig file for the `kube-controller-manager` service: Generate a kubeconfig file for the `kube-controller-manager` service:
``` ```
{ kubectl config set-cluster kubernetes-the-hard-way \
kubectl config set-cluster kubernetes-the-hard-way \ --certificate-authority=ca.pem \
--certificate-authority=ca.pem \ --embed-certs=true \
--embed-certs=true \ --server=https://127.0.0.1:6443 \
--server=https://127.0.0.1:6443 \ --kubeconfig=kube-controller-manager.kubeconfig
--kubeconfig=kube-controller-manager.kubeconfig
kubectl config set-credentials system:kube-controller-manager \ kubectl config set-credentials system:kube-controller-manager \
--client-certificate=kube-controller-manager.pem \ --client-certificate=kube-controller-manager.pem \
--client-key=kube-controller-manager-key.pem \ --client-key=kube-controller-manager-key.pem \
--embed-certs=true \ --embed-certs=true \
--kubeconfig=kube-controller-manager.kubeconfig --kubeconfig=kube-controller-manager.kubeconfig
kubectl config set-context default \ kubectl config set-context default \
--cluster=kubernetes-the-hard-way \ --cluster=kubernetes-the-hard-way \
--user=system:kube-controller-manager \ --user=system:kube-controller-manager \
--kubeconfig=kube-controller-manager.kubeconfig --kubeconfig=kube-controller-manager.kubeconfig
kubectl config use-context default --kubeconfig=kube-controller-manager.kubeconfig kubectl config use-context default --kubeconfig=kube-controller-manager.kubeconfig
}
``` ```
Results: Results:
@ -127,26 +123,24 @@ kube-controller-manager.kubeconfig
Generate a kubeconfig file for the `kube-scheduler` service: Generate a kubeconfig file for the `kube-scheduler` service:
``` ```
{ kubectl config set-cluster kubernetes-the-hard-way \
kubectl config set-cluster kubernetes-the-hard-way \ --certificate-authority=ca.pem \
--certificate-authority=ca.pem \ --embed-certs=true \
--embed-certs=true \ --server=https://127.0.0.1:6443 \
--server=https://127.0.0.1:6443 \ --kubeconfig=kube-scheduler.kubeconfig
--kubeconfig=kube-scheduler.kubeconfig
kubectl config set-credentials system:kube-scheduler \ kubectl config set-credentials system:kube-scheduler \
--client-certificate=kube-scheduler.pem \ --client-certificate=kube-scheduler.pem \
--client-key=kube-scheduler-key.pem \ --client-key=kube-scheduler-key.pem \
--embed-certs=true \ --embed-certs=true \
--kubeconfig=kube-scheduler.kubeconfig --kubeconfig=kube-scheduler.kubeconfig
kubectl config set-context default \ kubectl config set-context default \
--cluster=kubernetes-the-hard-way \ --cluster=kubernetes-the-hard-way \
--user=system:kube-scheduler \ --user=system:kube-scheduler \
--kubeconfig=kube-scheduler.kubeconfig --kubeconfig=kube-scheduler.kubeconfig
kubectl config use-context default --kubeconfig=kube-scheduler.kubeconfig kubectl config use-context default --kubeconfig=kube-scheduler.kubeconfig
}
``` ```
Results: Results:
@ -160,26 +154,24 @@ kube-scheduler.kubeconfig
Generate a kubeconfig file for the `admin` user: Generate a kubeconfig file for the `admin` user:
``` ```
{ kubectl config set-cluster kubernetes-the-hard-way \
kubectl config set-cluster kubernetes-the-hard-way \ --certificate-authority=ca.pem \
--certificate-authority=ca.pem \ --embed-certs=true \
--embed-certs=true \ --server=https://127.0.0.1:6443 \
--server=https://127.0.0.1:6443 \ --kubeconfig=admin.kubeconfig
--kubeconfig=admin.kubeconfig
kubectl config set-credentials admin \ kubectl config set-credentials admin \
--client-certificate=admin.pem \ --client-certificate=admin.pem \
--client-key=admin-key.pem \ --client-key=admin-key.pem \
--embed-certs=true \ --embed-certs=true \
--kubeconfig=admin.kubeconfig --kubeconfig=admin.kubeconfig
kubectl config set-context default \ kubectl config set-context default \
--cluster=kubernetes-the-hard-way \ --cluster=kubernetes-the-hard-way \
--user=admin \ --user=admin \
--kubeconfig=admin.kubeconfig --kubeconfig=admin.kubeconfig
kubectl config use-context default --kubeconfig=admin.kubeconfig kubectl config use-context default --kubeconfig=admin.kubeconfig
}
``` ```
Results: Results:

20
docs/07-bootstrapping-etcd.md
View File

@ -28,19 +28,15 @@ wget -q --show-progress --https-only --timestamping \
Extract and install the `etcd` server and the `etcdctl` command line utility: Extract and install the `etcd` server and the `etcdctl` command line utility:
``` ```
{ tar -xvf etcd-v3.3.5-linux-amd64.tar.gz
tar -xvf etcd-v3.3.5-linux-amd64.tar.gz sudo mv etcd-v3.3.5-linux-amd64/etcd* /usr/local/bin/
sudo mv etcd-v3.3.5-linux-amd64/etcd* /usr/local/bin/
}
``` ```
### Configure the etcd Server ### Configure the etcd Server
``` ```
{ sudo mkdir -p /etc/etcd /var/lib/etcd
sudo mkdir -p /etc/etcd /var/lib/etcd sudo cp ca.pem kubernetes-key.pem kubernetes.pem /etc/etcd/
sudo cp ca.pem kubernetes-key.pem kubernetes.pem /etc/etcd/
}
``` ```
The instance internal IP address will be used to serve client requests and communicate with etcd cluster peers. Retrieve the internal IP address for the current compute instance: The instance internal IP address will be used to serve client requests and communicate with etcd cluster peers. Retrieve the internal IP address for the current compute instance:
@ -94,11 +90,9 @@ EOF
### Start the etcd Server ### Start the etcd Server
``` ```
{ sudo systemctl daemon-reload
sudo systemctl daemon-reload sudo systemctl enable etcd
sudo systemctl enable etcd sudo systemctl start etcd
sudo systemctl start etcd
}
``` ```
> Remember to run the above commands on each controller node: `controller-0`, `controller-1`, and `controller-2`. > Remember to run the above commands on each controller node: `controller-0`, `controller-1`, and `controller-2`.

77
docs/08-bootstrapping-kubernetes-controllers.md
View File

@ -37,22 +37,19 @@ wget -q --show-progress --https-only --timestamping \
Install the Kubernetes binaries: Install the Kubernetes binaries:
``` ```
{ chmod +x kube-apiserver kube-controller-manager kube-scheduler kubectl
chmod +x kube-apiserver kube-controller-manager kube-scheduler kubectl
sudo mv kube-apiserver kube-controller-manager kube-scheduler kubectl /usr/local/bin/ sudo mv kube-apiserver kube-controller-manager kube-scheduler kubectl /usr/local/bin/
}
``` ```
### Configure the Kubernetes API Server ### Configure the Kubernetes API Server
``` ```
{ sudo mkdir -p /var/lib/kubernetes/
sudo mkdir -p /var/lib/kubernetes/
sudo mv ca.pem ca-key.pem kubernetes-key.pem kubernetes.pem \ sudo mv ca.pem ca-key.pem kubernetes-key.pem kubernetes.pem \
service-account-key.pem service-account.pem \ service-account-key.pem service-account.pem \
encryption-config.yaml /var/lib/kubernetes/ encryption-config.yaml /var/lib/kubernetes/
}
``` ```
The instance internal IP address will be used to advertise the API Server to members of the cluster. Retrieve the internal IP address for the current compute instance: The instance internal IP address will be used to advertise the API Server to members of the cluster. Retrieve the internal IP address for the current compute instance:
@ -191,11 +188,11 @@ EOF
### Start the Controller Services ### Start the Controller Services
``` ```
{ sudo systemctl daemon-reload
sudo systemctl daemon-reload
sudo systemctl enable kube-apiserver kube-controller-manager kube-scheduler sudo systemctl enable kube-apiserver kube-controller-manager kube-scheduler
sudo systemctl start kube-apiserver kube-controller-manager kube-scheduler
} sudo systemctl start kube-apiserver kube-controller-manager kube-scheduler
``` ```
> Allow up to 10 seconds for the Kubernetes API Server to fully initialize. > Allow up to 10 seconds for the Kubernetes API Server to fully initialize.
@ -227,12 +224,10 @@ EOF
``` ```
``` ```
{ sudo mv kubernetes.default.svc.cluster.local \
sudo mv kubernetes.default.svc.cluster.local \ /etc/nginx/sites-available/kubernetes.default.svc.cluster.local
/etc/nginx/sites-available/kubernetes.default.svc.cluster.local
sudo ln -s /etc/nginx/sites-available/kubernetes.default.svc.cluster.local /etc/nginx/sites-enabled/ sudo ln -s /etc/nginx/sites-available/kubernetes.default.svc.cluster.local /etc/nginx/sites-enabled/
}
``` ```
``` ```
@ -347,33 +342,31 @@ In this section you will provision an external load balancer to front the Kubern
Create the external load balancer network resources: Create the external load balancer network resources:
``` ```
{ KUBERNETES_PUBLIC_ADDRESS=$(gcloud compute addresses describe kubernetes-the-hard-way \
KUBERNETES_PUBLIC_ADDRESS=$(gcloud compute addresses describe kubernetes-the-hard-way \ --region $(gcloud config get-value compute/region) \
--region $(gcloud config get-value compute/region) \ --format 'value(address)')
--format 'value(address)')
gcloud compute http-health-checks create kubernetes \ gcloud compute http-health-checks create kubernetes \
--description "Kubernetes Health Check" \ --description "Kubernetes Health Check" \
--host "kubernetes.default.svc.cluster.local" \ --host "kubernetes.default.svc.cluster.local" \
--request-path "/healthz" --request-path "/healthz"
gcloud compute firewall-rules create kubernetes-the-hard-way-allow-health-check \ gcloud compute firewall-rules create kubernetes-the-hard-way-allow-health-check \
--network kubernetes-the-hard-way \ --network kubernetes-the-hard-way \
--source-ranges 209.85.152.0/22,209.85.204.0/22,35.191.0.0/16 \ --source-ranges 209.85.152.0/22,209.85.204.0/22,35.191.0.0/16 \
--allow tcp --allow tcp
gcloud compute target-pools create kubernetes-target-pool \ gcloud compute target-pools create kubernetes-target-pool \
--http-health-check kubernetes --http-health-check kubernetes
gcloud compute target-pools add-instances kubernetes-target-pool \ gcloud compute target-pools add-instances kubernetes-target-pool \
--instances controller-0,controller-1,controller-2 --instances controller-0,controller-1,controller-2
gcloud compute forwarding-rules create kubernetes-forwarding-rule \ gcloud compute forwarding-rules create kubernetes-forwarding-rule \
--address ${KUBERNETES_PUBLIC_ADDRESS} \ --address ${KUBERNETES_PUBLIC_ADDRESS} \
--ports 6443 \ --ports 6443 \
--region $(gcloud config get-value compute/region) \ --region $(gcloud config get-value compute/region) \
--target-pool kubernetes-target-pool --target-pool kubernetes-target-pool
}
``` ```
### Verification ### Verification

46
docs/09-bootstrapping-kubernetes-workers.md
View File

@ -19,10 +19,9 @@ gcloud compute ssh worker-0
Install the OS dependencies: Install the OS dependencies:
``` ```
{ sudo apt-get update
sudo apt-get update
sudo apt-get -y install socat conntrack ipset sudo apt-get -y install socat conntrack ipset
}
``` ```
> The socat binary enables support for the `kubectl port-forward` command. > The socat binary enables support for the `kubectl port-forward` command.
@ -56,14 +55,17 @@ sudo mkdir -p \
Install the worker binaries: Install the worker binaries:
``` ```
{ chmod +x kubectl kube-proxy kubelet runc.amd64 runsc
chmod +x kubectl kube-proxy kubelet runc.amd64 runsc
sudo mv runc.amd64 runc sudo mv runc.amd64 runc
sudo mv kubectl kube-proxy kubelet runc runsc /usr/local/bin/
sudo tar -xvf crictl-v1.0.0-beta.0-linux-amd64.tar.gz -C /usr/local/bin/ sudo mv kubectl kube-proxy kubelet runc runsc /usr/local/bin/
sudo tar -xvf cni-plugins-amd64-v0.6.0.tgz -C /opt/cni/bin/
sudo tar -xvf containerd-1.1.0.linux-amd64.tar.gz -C / sudo tar -xvf crictl-v1.0.0-beta.0-linux-amd64.tar.gz -C /usr/local/bin/
}
sudo tar -xvf cni-plugins-amd64-v0.6.0.tgz -C /opt/cni/bin/
sudo tar -xvf containerd-1.1.0.linux-amd64.tar.gz -C /
``` ```
### Configure CNI Networking ### Configure CNI Networking
@ -163,11 +165,11 @@ EOF
### Configure the Kubelet ### Configure the Kubelet
``` ```
{ sudo mv ${HOSTNAME}-key.pem ${HOSTNAME}.pem /var/lib/kubelet/
sudo mv ${HOSTNAME}-key.pem ${HOSTNAME}.pem /var/lib/kubelet/
sudo mv ${HOSTNAME}.kubeconfig /var/lib/kubelet/kubeconfig sudo mv ${HOSTNAME}.kubeconfig /var/lib/kubelet/kubeconfig
sudo mv ca.pem /var/lib/kubernetes/
} sudo mv ca.pem /var/lib/kubernetes/
``` ```
Create the `kubelet-config.yaml` configuration file: Create the `kubelet-config.yaml` configuration file:
@ -264,11 +266,11 @@ EOF
### Start the Worker Services ### Start the Worker Services
``` ```
{ sudo systemctl daemon-reload
sudo systemctl daemon-reload
sudo systemctl enable containerd kubelet kube-proxy sudo systemctl enable containerd kubelet kube-proxy
sudo systemctl start containerd kubelet kube-proxy
} sudo systemctl start containerd kubelet kube-proxy
``` ```
> Remember to run the above commands on each worker node: `worker-0`, `worker-1`, and `worker-2`. > Remember to run the above commands on each worker node: `worker-0`, `worker-1`, and `worker-2`.

30
docs/10-configuring-kubectl.md
View File

@ -11,26 +11,24 @@ Each kubeconfig requires a Kubernetes API Server to connect to. To support high
Generate a kubeconfig file suitable for authenticating as the `admin` user: Generate a kubeconfig file suitable for authenticating as the `admin` user:
``` ```
{ KUBERNETES_PUBLIC_ADDRESS=$(gcloud compute addresses describe kubernetes-the-hard-way \
KUBERNETES_PUBLIC_ADDRESS=$(gcloud compute addresses describe kubernetes-the-hard-way \ --region $(gcloud config get-value compute/region) \
--region $(gcloud config get-value compute/region) \ --format 'value(address)')
--format 'value(address)')
kubectl config set-cluster kubernetes-the-hard-way \ kubectl config set-cluster kubernetes-the-hard-way \
--certificate-authority=ca.pem \ --certificate-authority=ca.pem \
--embed-certs=true \ --embed-certs=true \
--server=https://${KUBERNETES_PUBLIC_ADDRESS}:6443 --server=https://${KUBERNETES_PUBLIC_ADDRESS}:6443
kubectl config set-credentials admin \ kubectl config set-credentials admin \
--client-certificate=admin.pem \ --client-certificate=admin.pem \
--client-key=admin-key.pem --client-key=admin-key.pem
kubectl config set-context kubernetes-the-hard-way \ kubectl config set-context kubernetes-the-hard-way \
--cluster=kubernetes-the-hard-way \ --cluster=kubernetes-the-hard-way \
--user=admin --user=admin
kubectl config use-context kubernetes-the-hard-way kubectl config use-context kubernetes-the-hard-way
}
``` ```
## Verification ## Verification

26
docs/14-cleanup.md
View File

@ -17,16 +17,14 @@ gcloud -q compute instances delete \
Delete the external load balancer network resources: Delete the external load balancer network resources:
``` ```
{ gcloud -q compute forwarding-rules delete kubernetes-forwarding-rule \
gcloud -q compute forwarding-rules delete kubernetes-forwarding-rule \ --region $(gcloud config get-value compute/region)
--region $(gcloud config get-value compute/region)
gcloud -q compute target-pools delete kubernetes-target-pool gcloud -q compute target-pools delete kubernetes-target-pool
gcloud -q compute http-health-checks delete kubernetes gcloud -q compute http-health-checks delete kubernetes
gcloud -q compute addresses delete kubernetes-the-hard-way gcloud -q compute addresses delete kubernetes-the-hard-way
}
``` ```
Delete the `kubernetes-the-hard-way` firewall rules: Delete the `kubernetes-the-hard-way` firewall rules:
@ -42,14 +40,12 @@ gcloud -q compute firewall-rules delete \
Delete the `kubernetes-the-hard-way` network VPC: Delete the `kubernetes-the-hard-way` network VPC:
``` ```
{ gcloud -q compute routes delete \
gcloud -q compute routes delete \ kubernetes-route-10-200-0-0-24 \
kubernetes-route-10-200-0-0-24 \ kubernetes-route-10-200-1-0-24 \
kubernetes-route-10-200-1-0-24 \ kubernetes-route-10-200-2-0-24
kubernetes-route-10-200-2-0-24
gcloud -q compute networks subnets delete kubernetes gcloud -q compute networks subnets delete kubernetes
gcloud -q compute networks delete kubernetes-the-hard-way gcloud -q compute networks delete kubernetes-the-hard-way
}
``` ```