Upgrade/1.24 (#291)

* Set up Vagrantfile
- Use Ubuntu 22.04
- Set required kernel parameters and tunables
- Optimise file for DRY by use of local functions
- No longer install Docker

* Update prerequisites

* Update compute resources

* Update client-tools

* Update cert authority

* Update kube config files

* Update sata encryption keys

* Update etcd

* Cert enhancements
- Use dig for host IPs
- Create front-proxy keys

* Update prereqs with lab defaults

* Minor update

* Dynamic kubelet reconfig removed in 1.24

* Update failed provisioning

* Update cert sujects. Use vars for IP addresses

* Use vars for IP addresses

* USe vars for IPs. Update unit file

* Unit updates for 1.24. Use vars for IPs

* 1.24 changes
- Update unit files
- Use vars for IPs
- Install containerd

* Use vars for IPs. Update outputs

* Remove CNI plugins - done earlier

* Update API versions

* Adjust VM RAM

* Update coredns version and api versions

* Update git ignore and attributes

* Note about deprecation warning

* Fix kubeconfig name

* Formatting changes + pin nginx version

* Update kubetest

* Update README

* Discuss why only 2 masters

* Note on changing service cidr range vs coredns

* Add RAM column to VM table

* Best practice - secure PKI

* Secure kubeconfig

* Add prev link

* Adding `Prev` links

* Squashed commit of the following:

commit 8fbd36069cbf7365f627e5ebf5a04e37cde085d9
Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com>
Date:   Thu Aug 25 20:06:10 2022 +0100

    Update dns-addon test

commit 5528e873ecbe3265155da48d24c24d696635af52
Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com>
Date:   Thu Aug 25 20:00:48 2022 +0100

    Fix get nodes

commit 0d88ab0d1c4b6a7ae05bc2552366460f741bb763
Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com>
Date:   Thu Aug 25 20:00:19 2022 +0100

    Fix env var name

commit e564db03ff9c4c9ef536bcc5cd999fa1e6a3de15
Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com>
Date:   Thu Aug 25 19:42:52 2022 +0100

    Update e2e-tests

commit 247a59f2c5b84e34972f396cf87a34bcbeb2d2ef
Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com>
Date:   Thu Aug 25 19:39:54 2022 +0100

    Updated e2e-tests

commit 60b33d025bb252570f41c13f90955ec8d59141a7
Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com>
Date:   Thu Aug 25 19:38:02 2022 +0100

    bashify commands in ```

commit 2814949d6dd569c59ea7ec61135784d51ad4de1f
Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com>
Date:   Thu Aug 25 19:35:32 2022 +0100

    Note deprecation warning when deploying weave

commit af0264e13e5f0e277f8f31e5115a813680aadd74
Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com>
Date:   Thu Aug 25 19:33:55 2022 +0100

    Nodes are ready at end of step 11

commit 050502386d36a8593ed7348e902cdff9ad9c64b2
Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com>
Date:   Thu Aug 25 19:30:00 2022 +0100

    Minor change CNI

commit 04bdc1483e9696ed018ac26b6480237ee1dcf1d1
Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com>
Date:   Thu Aug 25 19:21:22 2022 +0100

    Explain data at rest is in etcd

commit 243154b9866f5a7a1a49037f97e38c6bf7ffbcb7
Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com>
Date:   Thu Aug 25 19:18:49 2022 +0100

    Explanation of api cluster ip

commit dd168ac2e128cbd405248115d8724498fa18fa67
Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com>
Date:   Thu Aug 25 19:14:42 2022 +0100

    Include vagrant password

commit d51c65a77ac192e2468d92f0067958c69057a2e0
Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com>
Date:   Thu Aug 25 19:12:34 2022 +0100

    Update tmux message

commit 10f41737100ab410adb6b20712ee32cd80618e3d
Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com>
Date:   Thu Aug 25 19:09:23 2022 +0100

    Insert step to configure CNI on both workers
    Optionally with tmux

commit 8fd873f1492f6ea1c846b3309f57740e8501adee
Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com>
Date:   Thu Aug 25 18:42:27 2022 +0100

    Shuffle up to make room for common cni install

commit d650443b069a7543cbb4cf449818a81d84932007
Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com>
Date:   Thu Aug 25 07:34:59 2022 +0100

    Added warning output to componentstatuses

commit 7bfef8f16bd1a126dcf3e5f43a02d79517d64c74
Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com>
Date:   Thu Aug 25 07:34:38 2022 +0100

    Rearrange text

commit b16b92bc6513cf355a41afa22ddfe2696142c28b
Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com>
Date:   Thu Aug 25 07:34:18 2022 +0100

    Minor wording change
    DNS arress is conventionally .10

commit 96c9d25663ce3d721e670262bb6858e9a7183873
Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com>
Date:   Thu Aug 25 07:32:24 2022 +0100

    Use shell vars for etcd addresses

commit c9e223fba5324a1c65d6f583cf9e739b8459df5d
Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com>
Date:   Thu Aug 25 07:31:58 2022 +0100

    Update on network defaults

commit 1cf98649df9410b8a7d14c68bcb17c24aa6a210a
Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com>
Date:   Thu Aug 25 07:05:38 2022 +0100

    Get and install correct CNI components

commit 311905fba72f4a48cde4a73c589daea9b76042b7
Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com>
Date:   Thu Aug 25 06:18:55 2022 +0100

    Update Approve CSR

commit 4c39c84c172fde8ab2aafc4ea38b050eb7f3019b
Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com>
Date:   Wed Aug 24 20:34:53 2022 +0100

    Moving certs out of service kuebeconfigs

* Squashed commit of the following:

commit 252cc335739e3c8007ab86c951222aba954d80f7
Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com>
Date:   Sun Aug 28 20:29:23 2022 +0100

    Update external links

commit 8091d1a13bc5a29654db2b8fecd55b8180bf8cab
Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com>
Date:   Sun Aug 28 20:28:14 2022 +0100

    Mac M1 note

commit 8b7e6065ffb74532b6ad7570a8c978addcc7fb66
Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com>
Date:   Sun Aug 28 20:03:11 2022 +0100

    Tweak order of commands e2e tests

commit 857d039dd1dff28e92d392ad6c5e40814a9eb054
Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com>
Date:   Sun Aug 28 20:02:51 2022 +0100

    Fixing kubecomfig checks

commit 26f42049bebd2d539406e6e16c51bb06441702f1
Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com>
Date:   Sun Aug 28 15:51:13 2022 +0100

    Updated cert_verify

commit 0df54e4c3499e6d79b836e1dfcf74eb9fdf196b1
Author: Alistair Mackay <34012094+fireflycons@users.noreply.github.com>
Date:   Sun Aug 28 09:09:14 2022 +0100

    Rewite cert_verify
    Round 1 certs and kubeconfigs

* Update README
- Insert CNI lab
- Correct CNI versions

* Automate hostfile network settings
Determine from interface address passed in.

* Update 01-prerequisites.md

* Update 01-prerequisites.md

Correct the default vm ip range

* Review updates. Issue 1

* Review updates. Issue 2

* Review updates. Issue 3
In actual fact, the base script is cert_verfiy.sh so the error is in the
link created by the provisioner. You'll see that the later labs all
refer to it with underscore.

* Review updates. Issue 5

* Review updates. Issue 6

* Review updates. Issue 7
I whip through the scripts so fast, that even if I had copied it twice
to my quick script, I didn't notice it say that the resource exists and
is unchanged!

* These certs already copied in step 4

* Formatting and command grouping

* Review updates. Step 11 cert_verify
Needs to be done after kublet starts as it is looking
for the auto-issued cert

* Group coomand batches

* Remove duplicate clusterrolebinding

* Extraction of scripts from md using tool
This uses markdown comments and ```bash fence
to determine what to extract and for which hosts

Fixed shell var bug in step 11

* Fixed typos

* Be specific that we're doing shutdown, not suspend

* Minor edits for clarity

* remove the extra \

* Rename step 9 to CRI, as that's what it actually is

* Disambiguate CRI vs CNI

* small fixes

Co-authored-by: Tej Singh Rana <58101587+Tej-Singh-Rana@users.noreply.github.com>
This commit is contained in:
Alistair Mackay
2022-09-20 07:17:00 +01:00
committed by GitHub
parent 6327752d82
commit dcddd3347f
36 changed files with 1666 additions and 1270 deletions

20
vagrant/README.md Normal file
View File

@@ -0,0 +1,20 @@
# Vagrant
This directory contains the configuration for the virtual machines we will use for the installation.
A few prerequisites are handled by the VM provisioning steps.
## Kernel Settings
1. Disable cgroups v2. I found that Kubernetes currently doesn't play nice with cgroups v2, therefore we need to set a kernel boot parameter in grub to switch back to v1.
1. Install the `br_netfilter` kernel module that permits kube-proxy to manipulate IP tables rules
1. Add the two tunables `net.bridge.bridge-nf-call-iptables=1` and `net.ipv4.ip_forward=1` also required for successful pod networking.
## DNS settings
1. Set the default DNS server to be Google, as we know this always works.
1. Set up `/etc/hosts` so that all the VMs can resolve each other
## Other settings
1. Install configs for `vim` and `tmux` on master-1

132
vagrant/Vagrantfile vendored
View File

@@ -6,11 +6,37 @@
NUM_MASTER_NODE = 2
NUM_WORKER_NODE = 2
IP_NW = "192.168.5."
IP_NW = "192.168.56."
MASTER_IP_START = 10
NODE_IP_START = 20
LB_IP_START = 30
# Sets up hosts file and DNS
def setup_dns(node)
# Set up /etc/hosts
node.vm.provision "setup-hosts", :type => "shell", :path => "ubuntu/vagrant/setup-hosts.sh" do |s|
s.args = ["enp0s8", node.vm.hostname]
end
# Set up DNS resolution
node.vm.provision "setup-dns", type: "shell", :path => "ubuntu/update-dns.sh"
end
# Runs provisioning steps that are required by masters and workers
def provision_kubernetes_node(node)
# Set up kernel parameters, modules and tunables
node.vm.provision "setup-kernel", :type => "shell", :path => "ubuntu/setup-kernel.sh"
# Restart
node.vm.provision :shell do |shell|
shell.privileged = true
shell.inline = "echo Rebooting"
shell.reboot = true
end
# Set up DNS
setup_dns node
# Install cert verification script
node.vm.provision "shell", inline: "ln -s /vagrant/ubuntu/cert_verify.sh /home/vagrant/cert_verify.sh"
end
# All Vagrant configuration is done below. The "2" in Vagrant.configure
# configures the configuration version (we support older styles for
# backwards compatibility). Please don't change it unless you know what
@@ -23,99 +49,63 @@ Vagrant.configure("2") do |config|
# Every Vagrant development environment requires a box. You can search for
# boxes at https://vagrantcloud.com/search.
# config.vm.box = "base"
config.vm.box = "ubuntu/bionic64"
config.vm.box = "ubuntu/jammy64"
# Disable automatic box update checking. If you disable this, then
# boxes will only be checked for updates when the user runs
# `vagrant box outdated`. This is not recommended.
config.vm.box_check_update = false
# Create a public network, which generally matched to bridged network.
# Bridged networks make the machine appear as another physical device on
# your network.
# config.vm.network "public_network"
# Share an additional folder to the guest VM. The first argument is
# the path on the host to the actual folder. The second argument is
# the path on the guest to mount the folder. And the optional third
# argument is a set of non-required options.
# config.vm.synced_folder "../data", "/vagrant_data"
# Provider-specific configuration so you can fine-tune various
# backing providers for Vagrant. These expose provider-specific options.
# Example for VirtualBox:
#
# config.vm.provider "virtualbox" do |vb|
# # Customize the amount of memory on the VM:
# vb.memory = "1024"
# end
#
# View the documentation for the provider you are using for more
# information on available options.
# Provision Master Nodes
(1..NUM_MASTER_NODE).each do |i|
config.vm.define "master-#{i}" do |node|
# Name shown in the GUI
node.vm.provider "virtualbox" do |vb|
vb.name = "kubernetes-ha-master-#{i}"
vb.memory = 2048
vb.cpus = 2
config.vm.define "master-#{i}" do |node|
# Name shown in the GUI
node.vm.provider "virtualbox" do |vb|
vb.name = "kubernetes-ha-master-#{i}"
if i == 1
vb.memory = 2048 # More needed to run e2e tests at end
else
vb.memory = 1024
end
node.vm.hostname = "master-#{i}"
node.vm.network :private_network, ip: IP_NW + "#{MASTER_IP_START + i}"
node.vm.network "forwarded_port", guest: 22, host: "#{2710 + i}"
node.vm.provision "setup-hosts", :type => "shell", :path => "ubuntu/vagrant/setup-hosts.sh" do |s|
s.args = ["enp0s8"]
end
node.vm.provision "setup-dns", type: "shell", :path => "ubuntu/update-dns.sh"
node.vm.provision "file", source: "./ubuntu/cert_verify.sh", destination: "$HOME/"
vb.cpus = 2
end
node.vm.hostname = "master-#{i}"
node.vm.network :private_network, ip: IP_NW + "#{MASTER_IP_START + i}"
node.vm.network "forwarded_port", guest: 22, host: "#{2710 + i}"
provision_kubernetes_node node
if i == 1
# Install (opinionated) configs for vim and tmux on master-1. These used by the author for CKA exam.
node.vm.provision "file", source: "./ubuntu/tmux.conf", destination: "$HOME/.tmux.conf"
node.vm.provision "file", source: "./ubuntu/vimrc", destination: "$HOME/.vimrc"
end
end
end
# Provision Load Balancer Node
config.vm.define "loadbalancer" do |node|
node.vm.provider "virtualbox" do |vb|
vb.name = "kubernetes-ha-lb"
vb.memory = 512
vb.cpus = 1
vb.name = "kubernetes-ha-lb"
vb.memory = 512
vb.cpus = 1
end
node.vm.hostname = "loadbalancer"
node.vm.network :private_network, ip: IP_NW + "#{LB_IP_START}"
node.vm.network "forwarded_port", guest: 22, host: 2730
node.vm.provision "setup-hosts", :type => "shell", :path => "ubuntu/vagrant/setup-hosts.sh" do |s|
s.args = ["enp0s8"]
end
node.vm.provision "setup-dns", type: "shell", :path => "ubuntu/update-dns.sh"
node.vm.network "forwarded_port", guest: 22, host: 2730
setup_dns node
end
# Provision Worker Nodes
(1..NUM_WORKER_NODE).each do |i|
config.vm.define "worker-#{i}" do |node|
node.vm.provider "virtualbox" do |vb|
vb.name = "kubernetes-ha-worker-#{i}"
vb.memory = 512
vb.cpus = 1
end
node.vm.hostname = "worker-#{i}"
node.vm.network :private_network, ip: IP_NW + "#{NODE_IP_START + i}"
node.vm.network "forwarded_port", guest: 22, host: "#{2720 + i}"
node.vm.provision "setup-hosts", :type => "shell", :path => "ubuntu/vagrant/setup-hosts.sh" do |s|
s.args = ["enp0s8"]
end
node.vm.provision "setup-dns", type: "shell", :path => "ubuntu/update-dns.sh"
node.vm.provision "install-docker", type: "shell", :path => "ubuntu/install-docker-2.sh"
node.vm.provision "allow-bridge-nf-traffic", :type => "shell", :path => "ubuntu/allow-bridge-nf-traffic.sh"
node.vm.provision "file", source: "./ubuntu/cert_verify.sh", destination: "$HOME/"
node.vm.provider "virtualbox" do |vb|
vb.name = "kubernetes-ha-worker-#{i}"
vb.memory = 1024
vb.cpus = 1
end
node.vm.hostname = "worker-#{i}"
node.vm.network :private_network, ip: IP_NW + "#{NODE_IP_START + i}"
node.vm.network "forwarded_port", guest: 22, host: "#{2720 + i}"
provision_kubernetes_node node
end
end
end

View File

@@ -1,2 +0,0 @@
#!/bin/bash
sysctl net.bridge.bridge-nf-call-iptables=1

View File

@@ -4,19 +4,23 @@ set -e
# Green & Red marking for Success and Failed messages
SUCCESS='\033[0;32m'
FAILED='\033[0;31m'
FAILED='\033[0;31;1m'
NC='\033[0m'
# All Cert Location
# IP addresses
INTERNAL_IP=$(ip addr show enp0s8 | grep "inet " | awk '{print $2}' | cut -d / -f 1)
MASTER_1=$(dig +short master-1)
MASTER_2=$(dig +short master-2)
WORKER_1=$(dig +short worker-1)
WORKER_2=$(dig +short worker-2)
LOADBALANCER=$(dig +short loadbalancer)
LOCALHOST="127.0.0.1"
# All Cert Location
# ca certificate location
CACERT=ca.crt
CAKEY=ca.key
# admin certificate location
ADMINCERT=admin.crt
ADMINKEY=admin.key
# Kube controller manager certificate location
KCMCERT=kube-controller-manager.crt
KCMKEY=kube-controller-manager.key
@@ -91,294 +95,70 @@ SYSTEMD_WORKER_1_KP=/etc/systemd/system/kube-proxy.service
# Function - Master node #
check_cert_ca()
check_cert_and_key()
{
if [ -z $CACERT ] && [ -z $CAKEY ]
local name=$1
local subject=$2
local issuer=$3
local nokey=
local cert="${CERT_LOCATION}/$1.crt"
local key="${CERT_LOCATION}/$1.key"
if [ -z $cert -o -z $key ]
then
printf "${FAILED}please specify cert and key location\n"
printf "${FAILED}cert and/or key not present in ${CERT_LOCATION}. Perhaps you missed a copy step\n${NC}"
exit 1
elif [ -f $CACERT ] && [ -f $CAKEY ]
elif [ -f $cert -a -f $key ]
then
printf "${NC}CA cert and key found, verifying the authenticity\n"
CACERT_SUBJECT=$(openssl x509 -in $CACERT -text | grep "Subject: CN"| tr -d " ")
CACERT_ISSUER=$(openssl x509 -in $CACERT -text | grep "Issuer: CN"| tr -d " ")
CACERT_MD5=$(openssl x509 -noout -modulus -in $CACERT | openssl md5| awk '{print $2}')
CAKEY_MD5=$(openssl rsa -noout -modulus -in $CAKEY | openssl md5| awk '{print $2}')
if [ $CACERT_SUBJECT == "Subject:CN=KUBERNETES-CA" ] && [ $CACERT_ISSUER == "Issuer:CN=KUBERNETES-CA" ] && [ $CACERT_MD5 == $CAKEY_MD5 ]
printf "${NC}${name} cert and key found, verifying the authenticity\n"
CERT_SUBJECT=$(sudo openssl x509 -in $cert -text | grep "Subject: CN"| tr -d " ")
CERT_ISSUER=$(sudo openssl x509 -in $cert -text | grep "Issuer: CN"| tr -d " ")
CERT_MD5=$(sudo openssl x509 -noout -modulus -in $cert | openssl md5| awk '{print $2}')
KEY_MD5=$(sudo openssl rsa -noout -modulus -in $key | openssl md5| awk '{print $2}')
if [ $CERT_SUBJECT == "${subject}" ] && [ $CERT_ISSUER == "${issuer}" ] && [ $CERT_MD5 == $KEY_MD5 ]
then
printf "${SUCCESS}CA cert and key are correct\n"
printf "${SUCCESS}${name} cert and key are correct\n${NC}"
else
printf "${FAILED}Exiting...Found mismtach in the CA certificate and keys, More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/04-certificate-authority.md#certificate-authority\n"
printf "${FAILED}Exiting...Found mismtach in the ${name} certificate and keys, More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/04-certificate-authority.md#certificate-authority\n${NC}"
exit 1
fi
else
printf "${FAILED}ca.crt / ca.key is missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/04-certificate-authority.md#certificate-authority\n"
printf "${FAILED}${cert} / ${key} is missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/04-certificate-authority.md#certificate-authority\n"
echo "These should be in /var/lib/kubernetes/pki (most certs), /etc/etcd (eccd server certs) or /var/lib/kubelet (kubelet certs)${NC}"
exit 1
fi
}
check_cert_admin()
check_cert_only()
{
if [ -z $ADMINCERT ] && [ -z $ADMINKEY ]
local name=$1
local subject=$2
local issuer=$3
local cert="${CERT_LOCATION}/$1.crt"
# Worker-2 auto cert is a .pem
[ -f "${CERT_LOCATION}/$1.pem" ] && cert="${CERT_LOCATION}/$1.pem"
if [ -z $cert ]
then
printf "${FAILED}please specify cert and key location\n"
printf "${FAILED}cert not present in ${CERT_LOCATION}. Perhaps you missed a copy step\n${NC}"
exit 1
elif [ -f $ADMINCERT ] && [ -f $ADMINKEY ]
elif [ -f $cert ]
then
printf "${NC}admin cert and key found, verifying the authenticity\n"
ADMINCERT_SUBJECT=$(openssl x509 -in $ADMINCERT -text | grep "Subject: CN"| tr -d " ")
ADMINCERT_ISSUER=$(openssl x509 -in $ADMINCERT -text | grep "Issuer: CN"| tr -d " ")
ADMINCERT_MD5=$(openssl x509 -noout -modulus -in $ADMINCERT | openssl md5| awk '{print $2}')
ADMINKEY_MD5=$(openssl rsa -noout -modulus -in $ADMINKEY | openssl md5| awk '{print $2}')
if [ $ADMINCERT_SUBJECT == "Subject:CN=admin,O=system:masters" ] && [ $ADMINCERT_ISSUER == "Issuer:CN=KUBERNETES-CA" ] && [ $ADMINCERT_MD5 == $ADMINKEY_MD5 ]
printf "${NC}${name} cert found, verifying the authenticity\n"
CERT_SUBJECT=$(sudo openssl x509 -in $cert -text | grep "Subject: "| tr -d " ")
CERT_ISSUER=$(sudo openssl x509 -in $cert -text | grep "Issuer: CN"| tr -d " ")
CERT_MD5=$(sudo openssl x509 -noout -modulus -in $cert | openssl md5| awk '{print $2}')
if [ $CERT_SUBJECT == "${subject}" ] && [ $CERT_ISSUER == "${issuer}" ]
then
printf "${SUCCESS}admin cert and key are correct\n"
printf "${SUCCESS}${name} cert is correct\n${NC}"
else
printf "${FAILED}Exiting...Found mismtach in the admin certificate and keys, More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/04-certificate-authority.md#the-admin-client-certificate\n"
printf "${FAILED}Exiting...Found mismtach in the ${name} certificate, More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/04-certificate-authority.md#certificate-authority\n${NC}"
exit 1
fi
else
printf "${FAILED}admin.crt / admin.key is missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/04-certificate-authority.md#the-admin-client-certificate\n"
exit 1
fi
}
check_cert_kcm()
{
if [ -z $KCMCERT ] && [ -z $KCMKEY ]
then
printf "${FAILED}please specify cert and key location\n"
exit 1
elif [ -f $KCMCERT ] && [ -f $KCMKEY ]
then
printf "${NC}kube-controller-manager cert and key found, verifying the authenticity\n"
KCMCERT_SUBJECT=$(openssl x509 -in $KCMCERT -text | grep "Subject: CN"| tr -d " ")
KCMCERT_ISSUER=$(openssl x509 -in $KCMCERT -text | grep "Issuer: CN"| tr -d " ")
KCMCERT_MD5=$(openssl x509 -noout -modulus -in $KCMCERT | openssl md5| awk '{print $2}')
KCMKEY_MD5=$(openssl rsa -noout -modulus -in $KCMKEY | openssl md5| awk '{print $2}')
if [ $KCMCERT_SUBJECT == "Subject:CN=system:kube-controller-manager" ] && [ $KCMCERT_ISSUER == "Issuer:CN=KUBERNETES-CA" ] && [ $KCMCERT_MD5 == $KCMKEY_MD5 ]
then
printf "${SUCCESS}kube-controller-manager cert and key are correct\n"
else
printf "${FAILED}Exiting...Found mismtach in the kube-controller-manager certificate and keys, More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/04-certificate-authority.md#the-controller-manager-client-certificate\n"
exit 1
fi
else
printf "${FAILED}kube-controller-manager.crt / kube-controller-manager.key is missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/04-certificate-authority.md#the-controller-manager-client-certificate\n"
exit 1
fi
}
check_cert_kp()
{
if [ -z $KPCERT ] && [ -z $KPKEY ]
then
printf "${FAILED}please specify cert and key location\n"
exit 1
elif [ -f $KPCERT ] && [ -f $KPKEY ]
then
printf "${NC}kube-proxy cert and key found, verifying the authenticity\n"
KPCERT_SUBJECT=$(openssl x509 -in $KPCERT -text | grep "Subject: CN"| tr -d " ")
KPCERT_ISSUER=$(openssl x509 -in $KPCERT -text | grep "Issuer: CN"| tr -d " ")
KPCERT_MD5=$(openssl x509 -noout -modulus -in $KPCERT | openssl md5| awk '{print $2}')
KPKEY_MD5=$(openssl rsa -noout -modulus -in $KPKEY | openssl md5| awk '{print $2}')
if [ $KPCERT_SUBJECT == "Subject:CN=system:kube-proxy" ] && [ $KPCERT_ISSUER == "Issuer:CN=KUBERNETES-CA" ] && [ $KPCERT_MD5 == $KPKEY_MD5 ]
then
printf "${SUCCESS}kube-proxy cert and key are correct\n"
else
printf "${FAILED}Exiting...Found mismtach in the kube-proxy certificate and keys, More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/04-certificate-authority.md#the-kube-proxy-client-certificate\n"
exit 1
fi
else
printf "${FAILED}kube-proxy.crt / kube-proxy.key is missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/04-certificate-authority.md#the-kube-proxy-client-certificate\n"
exit 1
fi
}
check_cert_ks()
{
if [ -z $KSCERT ] && [ -z $KSKEY ]
then
printf "${FAILED}please specify cert and key location\n"
exit 1
elif [ -f $KSCERT ] && [ -f $KSKEY ]
then
printf "${NC}kube-scheduler cert and key found, verifying the authenticity\n"
KSCERT_SUBJECT=$(openssl x509 -in $KSCERT -text | grep "Subject: CN"| tr -d " ")
KSCERT_ISSUER=$(openssl x509 -in $KSCERT -text | grep "Issuer: CN"| tr -d " ")
KSCERT_MD5=$(openssl x509 -noout -modulus -in $KSCERT | openssl md5| awk '{print $2}')
KSKEY_MD5=$(openssl rsa -noout -modulus -in $KSKEY | openssl md5| awk '{print $2}')
if [ $KSCERT_SUBJECT == "Subject:CN=system:kube-scheduler" ] && [ $KSCERT_ISSUER == "Issuer:CN=KUBERNETES-CA" ] && [ $KSCERT_MD5 == $KSKEY_MD5 ]
then
printf "${SUCCESS}kube-scheduler cert and key are correct\n"
else
printf "${FAILED}Exiting...Found mismtach in the kube-scheduler certificate and keys, More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/04-certificate-authority.md#the-scheduler-client-certificate\n"
exit 1
fi
else
printf "${FAILED}kube-scheduler.crt / kube-scheduler.key is missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/04-certificate-authority.md#the-scheduler-client-certificate\n"
exit 1
fi
}
check_cert_api()
{
if [ -z $APICERT ] && [ -z $APIKEY ]
then
printf "${FAILED}please specify kube-api cert and key location, Exiting....\n"
exit 1
elif [ -f $APICERT ] && [ -f $APIKEY ]
then
printf "${NC}kube-apiserver cert and key found, verifying the authenticity\n"
APICERT_SUBJECT=$(openssl x509 -in $APICERT -text | grep "Subject: CN"| tr -d " ")
APICERT_ISSUER=$(openssl x509 -in $APICERT -text | grep "Issuer: CN"| tr -d " ")
APICERT_MD5=$(openssl x509 -noout -modulus -in $APICERT | openssl md5| awk '{print $2}')
APIKEY_MD5=$(openssl rsa -noout -modulus -in $APIKEY | openssl md5| awk '{print $2}')
if [ $APICERT_SUBJECT == "Subject:CN=kube-apiserver" ] && [ $APICERT_ISSUER == "Issuer:CN=KUBERNETES-CA" ] && [ $APICERT_MD5 == $APIKEY_MD5 ]
then
printf "${SUCCESS}kube-apiserver cert and key are correct\n"
else
printf "${FAILED}Exiting...Found mismtach in the kube-apiserver certificate and keys, More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/04-certificate-authority.md#the-kubernetes-api-server-certificate\n"
exit 1
fi
else
printf "${FAILED}kube-apiserver.crt / kube-apiserver.key is missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/04-certificate-authority.md#the-kubernetes-api-server-certificate\n"
exit 1
fi
}
check_cert_etcd()
{
if [ -z $ETCDCERT ] && [ -z $ETCDKEY ]
then
printf "${FAILED}please specify ETCD cert and key location, Exiting....\n"
exit 1
elif [ -f $ETCDCERT ] && [ -f $ETCDKEY ]
then
printf "${NC}ETCD cert and key found, verifying the authenticity\n"
ETCDCERT_SUBJECT=$(openssl x509 -in $ETCDCERT -text | grep "Subject: CN"| tr -d " ")
ETCDCERT_ISSUER=$(openssl x509 -in $ETCDCERT -text | grep "Issuer: CN"| tr -d " ")
ETCDCERT_MD5=$(openssl x509 -noout -modulus -in $ETCDCERT | openssl md5| awk '{print $2}')
ETCDKEY_MD5=$(openssl rsa -noout -modulus -in $ETCDKEY | openssl md5| awk '{print $2}')
if [ $ETCDCERT_SUBJECT == "Subject:CN=etcd-server" ] && [ $ETCDCERT_ISSUER == "Issuer:CN=KUBERNETES-CA" ] && [ $ETCDCERT_MD5 == $ETCDKEY_MD5 ]
then
printf "${SUCCESS}etcd-server.crt / etcd-server.key are correct\n"
else
printf "${FAILED}Exiting...Found mismtach in the ETCD certificate and keys, More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/04-certificate-authority.md#the-etcd-server-certificate\n"
exit 1
fi
else
printf "${FAILED}etcd-server.crt / etcd-server.key is missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/04-certificate-authority.md#the-etcd-server-certificate\n"
exit 1
fi
}
check_cert_sa()
{
if [ -z $SACERT ] && [ -z $SAKEY ]
then
printf "${FAILED}please specify Service Account cert and key location, Exiting....\n"
exit 1
elif [ -f $SACERT ] && [ -f $SAKEY ]
then
printf "${NC}service account cert and key found, verifying the authenticity\n"
SACERT_SUBJECT=$(openssl x509 -in $SACERT -text | grep "Subject: CN"| tr -d " ")
SACERT_ISSUER=$(openssl x509 -in $SACERT -text | grep "Issuer: CN"| tr -d " ")
SACERT_MD5=$(openssl x509 -noout -modulus -in $SACERT | openssl md5| awk '{print $2}')
SAKEY_MD5=$(openssl rsa -noout -modulus -in $SAKEY | openssl md5| awk '{print $2}')
if [ $SACERT_SUBJECT == "Subject:CN=service-accounts" ] && [ $SACERT_ISSUER == "Issuer:CN=KUBERNETES-CA" ] && [ $SACERT_MD5 == $SAKEY_MD5 ]
then
printf "${SUCCESS}Service Account cert and key are correct\n"
else
printf "${FAILED}Exiting...Found mismtach in the Service Account certificate and keys, More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/04-certificate-authority.md#the-service-account-key-pair\n"
exit 1
fi
else
printf "${FAILED}service-account.crt / service-account.key is missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/04-certificate-authority.md#the-service-account-key-pair\n"
exit 1
fi
}
check_cert_kpkubeconfig()
{
if [ -z $KPKUBECONFIG ]
then
printf "${FAILED}please specify kube-proxy kubeconfig location\n"
exit 1
elif [ -f $KPKUBECONFIG ]
then
printf "${NC}kube-proxy kubeconfig file found, verifying the authenticity\n"
KPKUBECONFIG_SUBJECT=$(cat $KPKUBECONFIG | grep "client-certificate-data:" | awk '{print $2}' | base64 --decode | openssl x509 -text | grep "Subject: CN" | tr -d " ")
KPKUBECONFIG_ISSUER=$(cat $KPKUBECONFIG | grep "client-certificate-data:" | awk '{print $2}' | base64 --decode | openssl x509 -text | grep "Issuer: CN" | tr -d " ")
KPKUBECONFIG_CERT_MD5=$(cat $KPKUBECONFIG | grep "client-certificate-data:" | awk '{print $2}' | base64 --decode | openssl x509 -noout | openssl md5 | awk '{print $2}')
KPKUBECONFIG_KEY_MD5=$(cat $KPKUBECONFIG | grep "client-key-data" | awk '{print $2}' | base64 --decode | openssl rsa -noout | openssl md5 | awk '{print $2}')
KPKUBECONFIG_SERVER=$(cat $KPKUBECONFIG | grep "server:"| awk '{print $2}')
if [ $KPKUBECONFIG_SUBJECT == "Subject:CN=system:kube-proxy" ] && [ $KPKUBECONFIG_ISSUER == "Issuer:CN=KUBERNETES-CA" ] && [ $KPKUBECONFIG_CERT_MD5 == $KPKUBECONFIG_KEY_MD5 ] && [ $KPKUBECONFIG_SERVER == "https://192.168.5.30:6443" ]
then
printf "${SUCCESS}kube-proxy kubeconfig cert and key are correct\n"
else
printf "${FAILED}Exiting...Found mismtach in the kube-proxy kubeconfig certificate and keys, More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/05-kubernetes-configuration-files.md#the-kube-proxy-kubernetes-configuration-file\n"
exit 1
fi
else
printf "${FAILED}kube-proxy kubeconfig file is missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/05-kubernetes-configuration-files.md#the-kube-proxy-kubernetes-configuration-file\n"
exit 1
fi
}
check_cert_kcmkubeconfig()
{
if [ -z $KCMKUBECONFIG ]
then
printf "${FAILED}please specify kube-controller-manager kubeconfig location\n"
exit 1
elif [ -f $KCMKUBECONFIG ]
then
printf "${NC}kube-controller-manager kubeconfig file found, verifying the authenticity\n"
KCMKUBECONFIG_SUBJECT=$(cat $KCMKUBECONFIG | grep "client-certificate-data:" | awk '{print $2}' | base64 --decode | openssl x509 -text | grep "Subject: CN" | tr -d " ")
KCMKUBECONFIG_ISSUER=$(cat $KCMKUBECONFIG | grep "client-certificate-data:" | awk '{print $2}' | base64 --decode | openssl x509 -text | grep "Issuer: CN" | tr -d " ")
KCMKUBECONFIG_CERT_MD5=$(cat $KCMKUBECONFIG | grep "client-certificate-data:" | awk '{print $2}' | base64 --decode | openssl x509 -noout | openssl md5 | awk '{print $2}')
KCMKUBECONFIG_KEY_MD5=$(cat $KCMKUBECONFIG | grep "client-key-data" | awk '{print $2}' | base64 --decode | openssl rsa -noout | openssl md5 | awk '{print $2}')
KCMKUBECONFIG_SERVER=$(cat $KCMKUBECONFIG | grep "server:"| awk '{print $2}')
if [ $KCMKUBECONFIG_SUBJECT == "Subject:CN=system:kube-controller-manager" ] && [ $KCMKUBECONFIG_ISSUER == "Issuer:CN=KUBERNETES-CA" ] && [ $KCMKUBECONFIG_CERT_MD5 == $KCMKUBECONFIG_KEY_MD5 ] && [ $KCMKUBECONFIG_SERVER == "https://127.0.0.1:6443" ]
then
printf "${SUCCESS}kube-controller-manager kubeconfig cert and key are correct\n"
else
printf "${FAILED}Exiting...Found mismtach in the kube-controller-manager kubeconfig certificate and keys, More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/05-kubernetes-configuration-files.md#the-kube-controller-manager-kubernetes-configuration-file\n"
exit 1
fi
else
printf "${FAILED}kube-controller-manager kubeconfig file is missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/05-kubernetes-configuration-files.md#the-kube-controller-manager-kubernetes-configuration-file\n"
exit 1
fi
}
check_cert_kskubeconfig()
{
if [ -z $KSKUBECONFIG ]
then
printf "${FAILED}please specify kube-scheduler kubeconfig location\n"
exit 1
elif [ -f $KSKUBECONFIG ]
then
printf "${NC}kube-scheduler kubeconfig file found, verifying the authenticity\n"
KSKUBECONFIG_SUBJECT=$(cat $KSKUBECONFIG | grep "client-certificate-data:" | awk '{print $2}' | base64 --decode | openssl x509 -text | grep "Subject: CN" | tr -d " ")
KSKUBECONFIG_ISSUER=$(cat $KSKUBECONFIG | grep "client-certificate-data:" | awk '{print $2}' | base64 --decode | openssl x509 -text | grep "Issuer: CN" | tr -d " ")
KSKUBECONFIG_CERT_MD5=$(cat $KSKUBECONFIG | grep "client-certificate-data:" | awk '{print $2}' | base64 --decode | openssl x509 -noout | openssl md5 | awk '{print $2}')
KSKUBECONFIG_KEY_MD5=$(cat $KSKUBECONFIG | grep "client-key-data" | awk '{print $2}' | base64 --decode | openssl rsa -noout | openssl md5 | awk '{print $2}')
KSKUBECONFIG_SERVER=$(cat $KSKUBECONFIG | grep "server:"| awk '{print $2}')
if [ $KSKUBECONFIG_SUBJECT == "Subject:CN=system:kube-scheduler" ] && [ $KSKUBECONFIG_ISSUER == "Issuer:CN=KUBERNETES-CA" ] && [ $KSKUBECONFIG_CERT_MD5 == $KSKUBECONFIG_KEY_MD5 ] && [ $KSKUBECONFIG_SERVER == "https://127.0.0.1:6443" ]
then
printf "${SUCCESS}kube-scheduler kubeconfig cert and key are correct\n"
else
printf "${FAILED}Exiting...Found mismtach in the kube-scheduler kubeconfig certificate and keys, More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/05-kubernetes-configuration-files.md#the-kube-scheduler-kubernetes-configuration-file\n"
exit 1
fi
else
printf "${FAILED}kube-scheduler kubeconfig file is missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/05-kubernetes-configuration-files.md#the-kube-scheduler-kubernetes-configuration-file\n"
printf "${FAILED}${cert} missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/04-certificate-authority.md#certificate-authority\n${NC}"
echo "These should be in ${CERT_LOCATION}${NC}"
exit 1
fi
}
@@ -387,17 +167,17 @@ check_cert_adminkubeconfig()
{
if [ -z $ADMINKUBECONFIG ]
then
printf "${FAILED}please specify admin kubeconfig location\n"
printf "${FAILED}please specify admin kubeconfig location\n${NC}"
exit 1
elif [ -f $ADMINKUBECONFIG ]
then
printf "${NC}admin kubeconfig file found, verifying the authenticity\n"
ADMINKUBECONFIG_SUBJECT=$(cat $ADMINKUBECONFIG | grep "client-certificate-data:" | awk '{print $2}' | base64 --decode | openssl x509 -text | grep "Subject: CN" | tr -d " ")
ADMINKUBECONFIG_ISSUER=$(cat $ADMINKUBECONFIG | grep "client-certificate-data:" | awk '{print $2}' | base64 --decode | openssl x509 -text | grep "Issuer: CN" | tr -d " ")
ADMINKUBECONFIG_CERT_MD5=$(cat $ADMINKUBECONFIG | grep "client-certificate-data:" | awk '{print $2}' | base64 --decode | openssl x509 -noout | openssl md5 | awk '{print $2}')
ADMINKUBECONFIG_SUBJECT=$(cat $ADMINKUBECONFIG | grep "client-certificate-data:" | awk '{print $2}' | base64 --decode | sudo openssl x509 -text | grep "Subject: CN" | tr -d " ")
ADMINKUBECONFIG_ISSUER=$(cat $ADMINKUBECONFIG | grep "client-certificate-data:" | awk '{print $2}' | base64 --decode | sudo openssl x509 -text | grep "Issuer: CN" | tr -d " ")
ADMINKUBECONFIG_CERT_MD5=$(cat $ADMINKUBECONFIG | grep "client-certificate-data:" | awk '{print $2}' | base64 --decode | sudo openssl x509 -noout | openssl md5 | awk '{print $2}')
ADMINKUBECONFIG_KEY_MD5=$(cat $ADMINKUBECONFIG | grep "client-key-data" | awk '{print $2}' | base64 --decode | openssl rsa -noout | openssl md5 | awk '{print $2}')
ADMINKUBECONFIG_SERVER=$(cat $ADMINKUBECONFIG | grep "server:"| awk '{print $2}')
if [ $ADMINKUBECONFIG_SUBJECT == "Subject:CN=admin,O=system:masters" ] && [ $ADMINKUBECONFIG_ISSUER == "Issuer:CN=KUBERNETES-CA" ] && [ $ADMINKUBECONFIG_CERT_MD5 == $ADMINKUBECONFIG_KEY_MD5 ] && [ $ADMINKUBECONFIG_SERVER == "https://127.0.0.1:6443" ]
if [ $ADMINKUBECONFIG_SUBJECT == "Subject:CN=admin,O=system:masters" ] && [ $ADMINKUBECONFIG_ISSUER == "Issuer:CN=KUBERNETES-CA,O=Kubernetes" ] && [ $ADMINKUBECONFIG_CERT_MD5 == $ADMINKUBECONFIG_KEY_MD5 ] && [ $ADMINKUBECONFIG_SERVER == "https://127.0.0.1:6443" ]
then
printf "${SUCCESS}admin kubeconfig cert and key are correct\n"
else
@@ -410,11 +190,81 @@ check_cert_adminkubeconfig()
fi
}
get_kubeconfig_cert_path()
{
local kubeconfig=$1
local cert_field=$2
sudo cat $kubeconfig | grep cert_field | awk '{print $2}'
}
check_kubeconfig()
{
local name=$1
local location=$2
local apiserver=$3
local kubeconfig="${location}/${name}.kubeconfig"
echo "Checking $kubeconfig"
check_kubeconfig_exists $name $location
ca=$(get_kubeconfig_cert_path $kubeconfig "certificate-authority")
cert=$(get_kubeconfig_cert_path $kubeconfig "client-certificate")
key=$(get_kubeconfig_cert_path $kubeconfig "client-key")
server=$(sudo cat $kubeconfig | grep server | awk '{print $2}')
if [ -f "$ca"]
then
printf "${SUCCESS}Path to CA certificate is correct${NC}\n"
else
printf "${FAIL}CA certificate not found at ${ca}${NC}\n"
exit 1
fi
if [ -f "$cert"]
then
printf "${SUCCESS}Path to client certificate is correct${NC}\n"
else
printf "${FAIL}Client certificate not found at ${cert}${NC}\n"
exit 1
fi
if [ -f "$key"]
then
printf "${SUCCESS}Path to client key is correct${NC}\n"
else
printf "${FAIL}Client key not found at ${key}${NC}\n"
exit 1
fi
if [ "$apiserver" = "$server" ]
then
printf "${SUCCESS}Server URL is correct${NC}\n"
else
printf "${FAIL}Server URL ${server} is incorrect${NC}\n"
exit 1
fi
}
check_kubeconfig_exists() {
local name=$1
local location=$2
local kubeconfig="${location}/${name}.kubeconfig"
if [ -f "${kubeconfig}" ]
then
printf "${SUCCESS}${kubeconfig} found${NC}\n"
else
printf "${FAIL}${kubeconfig} not found!${NC}\n"
exit 1
fi
}
check_systemd_etcd()
{
if [ -z $ETCDCERT ] && [ -z $ETCDKEY ]
then
printf "${FAILED}please specify ETCD cert and key location, Exiting....\n"
printf "${FAILED}please specify ETCD cert and key location, Exiting....\n${NC}"
exit 1
elif [ -f $SYSTEMD_ETCD_FILE ]
then
@@ -430,7 +280,7 @@ check_systemd_etcd()
PEER_TRUSTED_CA_FILE=$(systemctl cat etcd.service | grep "\--peer-trusted-ca-file"| awk '{print $1}'| cut -d "=" -f2)
# Systemd advertise , client and peer url's
INTERNAL_IP=$(ip addr show enp0s8 | grep "inet " | awk '{print $2}' | cut -d / -f 1)
IAP_URL=$(systemctl cat etcd.service | grep "\--initial-advertise-peer-urls"| awk '{print $2}')
LP_URL=$(systemctl cat etcd.service | grep "\--listen-peer-urls"| awk '{print $2}')
LC_URL=$(systemctl cat etcd.service | grep "\--listen-client-urls"| awk '{print $2}')
@@ -443,23 +293,23 @@ check_systemd_etcd()
if [ $CERT_FILE == $ETCDCERT ] && [ $KEY_FILE == $ETCDKEY ] && [ $PEER_CERT_FILE == $ETCDCERT ] && [ $PEER_KEY_FILE == $ETCDKEY ] && \
[ $TRUSTED_CA_FILE == $ETCD_CA_CERT ] && [ $PEER_TRUSTED_CA_FILE = $ETCD_CA_CERT ]
then
printf "${SUCCESS}ETCD certificate, ca and key files are correct under systemd service\n"
printf "${SUCCESS}ETCD certificate, ca and key files are correct under systemd service\n${NC}"
else
printf "${FAILED}Exiting...Found mismtach in the ETCD certificate, ca and keys. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/07-bootstrapping-etcd.md#configure-the-etcd-server\n"
printf "${FAILED}Exiting...Found mismtach in the ETCD certificate, ca and keys. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/07-bootstrapping-etcd.md#configure-the-etcd-server\n${NC}"
exit 1
fi
if [ $IAP_URL == "https://$INTERNAL_IP:2380" ] && [ $LP_URL == "https://$INTERNAL_IP:2380" ] && [ $LC_URL == "https://$INTERNAL_IP:2379,https://127.0.0.1:2379" ] && \
[ $AC_URL == "https://$INTERNAL_IP:2379" ]
then
printf "${SUCCESS}ETCD initial-advertise-peer-urls, listen-peer-urls, listen-client-urls, advertise-client-urls are correct\n"
printf "${SUCCESS}ETCD initial-advertise-peer-urls, listen-peer-urls, listen-client-urls, advertise-client-urls are correct\n${NC}"
else
printf "${FAILED}Exiting...Found mismtach in the ETCD initial-advertise-peer-urls / listen-peer-urls / listen-client-urls / advertise-client-urls. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/07-bootstrapping-etcd.md#configure-the-etcd-server\n"
printf "${FAILED}Exiting...Found mismtach in the ETCD initial-advertise-peer-urls / listen-peer-urls / listen-client-urls / advertise-client-urls. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/07-bootstrapping-etcd.md#configure-the-etcd-server\n${NC}"
exit 1
fi
else
printf "${FAILED}etcd-server.crt / etcd-server.key is missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/07-bootstrapping-etcd.md#configure-the-etcd-server\n"
printf "${FAILED}etcd-server.crt / etcd-server.key is missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/07-bootstrapping-etcd.md#configure-the-etcd-server\n${NC}"
exit 1
fi
}
@@ -468,13 +318,12 @@ check_systemd_api()
{
if [ -z $APICERT ] && [ -z $APIKEY ]
then
printf "${FAILED}please specify kube-api cert and key location, Exiting....\n"
printf "${FAILED}please specify kube-api cert and key location, Exiting....\n${NC}"
exit 1
elif [ -f $SYSTEMD_API_FILE ]
then
printf "${NC}Systemd for kube-api service found, verifying the authenticity\n"
printf "Systemd for kube-api service found, verifying the authenticity\n"
INTERNAL_IP=$(ip addr show enp0s8 | grep "inet " | awk '{print $2}' | cut -d / -f 1)
ADVERTISE_ADDRESS=$(systemctl cat kube-apiserver.service | grep "\--advertise-address" | awk '{print $1}' | cut -d "=" -f2)
CLIENT_CA_FILE=$(systemctl cat kube-apiserver.service | grep "\--client-ca-file" | awk '{print $1}' | cut -d "=" -f2)
ETCD_CA_FILE=$(systemctl cat kube-apiserver.service | grep "\--etcd-cafile" | awk '{print $1}' | cut -d "=" -f2)
@@ -487,41 +336,44 @@ check_systemd_api()
TLS_CERT_FILE=$(systemctl cat kube-apiserver.service | grep "\--tls-cert-file" | awk '{print $1}' | cut -d "=" -f2)
TLS_PRIVATE_KEY_FILE=$(systemctl cat kube-apiserver.service | grep "\--tls-private-key-file" | awk '{print $1}' | cut -d "=" -f2)
CACERT=/var/lib/kubernetes/ca.crt
APICERT=/var/lib/kubernetes/kube-apiserver.crt
APIKEY=/var/lib/kubernetes/kube-apiserver.key
SACERT=/var/lib/kubernetes/service-account.crt
PKI=/var/lib/kubernetes/pki
CACERT="${PKI}/ca.crt"
APICERT="${PKI}/kube-apiserver.crt"
APIKEY="${PKI}/kube-apiserver.key"
SACERT="${PKI}/service-account.crt"
KCCERT="${PKI}/apiserver-kubelet-client.crt"
KCKEY="${PKI}/apiserver-kubelet-client.key"
if [ $ADVERTISE_ADDRESS == $INTERNAL_IP ] && [ $CLIENT_CA_FILE == $CACERT ] && [ $ETCD_CA_FILE == $CACERT ] && \
[ $ETCD_CERT_FILE == "/var/lib/kubernetes/etcd-server.crt" ] && [ $ETCD_KEY_FILE == "/var/lib/kubernetes/etcd-server.key" ] && \
[ $KUBELET_CERTIFICATE_AUTHORITY == $CACERT ] && [ $KUBELET_CLIENT_CERTIFICATE == $APICERT ] && [ $KUBELET_CLIENT_KEY == $APIKEY ] && \
[ $ETCD_CERT_FILE == "${PKI}/etcd-server.crt" ] && [ $ETCD_KEY_FILE == "${PKI}/etcd-server.key" ] && \
[ $KUBELET_CERTIFICATE_AUTHORITY == $CACERT ] && [ $KUBELET_CLIENT_CERTIFICATE == $KCCERT ] && [ $KUBELET_CLIENT_KEY == $KCKEY ] && \
[ $SERVICE_ACCOUNT_KEY_FILE == $SACERT ] && [ $TLS_CERT_FILE == $APICERT ] && [ $TLS_PRIVATE_KEY_FILE == $APIKEY ]
then
printf "${SUCCESS}kube-apiserver advertise-address/ client-ca-file/ etcd-cafile/ etcd-certfile/ etcd-keyfile/ kubelet-certificate-authority/ kubelet-client-certificate/ kubelet-client-key/ service-account-key-file/ tls-cert-file/ tls-private-key-file are correct\n"
printf "${SUCCESS}kube-apiserver advertise-address/ client-ca-file/ etcd-cafile/ etcd-certfile/ etcd-keyfile/ kubelet-certificate-authority/ kubelet-client-certificate/ kubelet-client-key/ service-account-key-file/ tls-cert-file/ tls-private-key-file are correct\n${NC}"
else
printf "${FAILED}Exiting...Found mismtach in the kube-apiserver systemd file, check advertise-address/ client-ca-file/ etcd-cafile/ etcd-certfile/ etcd-keyfile/ kubelet-certificate-authority/ kubelet-client-certificate/ kubelet-client-key/ service-account-key-file/ tls-cert-file/ tls-private-key-file. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/08-bootstrapping-kubernetes-controllers.md#configure-the-kubernetes-api-server\n"
printf "${FAILED}Exiting...Found mismtach in the kube-apiserver systemd file, check advertise-address/ client-ca-file/ etcd-cafile/ etcd-certfile/ etcd-keyfile/ kubelet-certificate-authority/ kubelet-client-certificate/ kubelet-client-key/ service-account-key-file/ tls-cert-file/ tls-private-key-file. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/08-bootstrapping-kubernetes-controllers.md#configure-the-kubernetes-api-server\n${NC}"
exit 1
fi
else
printf "${FAILED}kube-apiserver.crt / kube-apiserver.key is missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/08-bootstrapping-kubernetes-controllers.md#configure-the-kubernetes-api-server\n"
printf "${FAILED}kube-apiserver.crt / kube-apiserver.key is missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/08-bootstrapping-kubernetes-controllers.md#configure-the-kubernetes-api-server\n${NC}"
exit 1
fi
}
check_systemd_kcm()
{
KCMCERT=/var/lib/kubernetes/kube-controller-manager.crt
KCMKEY=/var/lib/kubernetes/kube-controller-manager.key
CACERT=/var/lib/kubernetes/ca.crt
CAKEY=/var/lib/kubernetes/ca.key
SAKEY=/var/lib/kubernetes/service-account.key
KCMCERT=/var/lib/kubernetes/pki/kube-controller-manager.crt
KCMKEY=/var/lib/kubernetes/pki/kube-controller-manager.key
CACERT=/var/lib/kubernetes/pki/ca.crt
CAKEY=/var/lib/kubernetes/pki/ca.key
SAKEY=/var/lib/kubernetes/pki/service-account.key
KCMKUBECONFIG=/var/lib/kubernetes/kube-controller-manager.kubeconfig
if [ -z $KCMCERT ] && [ -z $KCMKEY ]
then
printf "${FAILED}please specify cert and key location\n"
printf "${FAILED}please specify cert and key location\n${NC}"
exit 1
elif [ -f $SYSTEMD_KCM_FILE ]
then
printf "${NC}Systemd for kube-controller-manager service found, verifying the authenticity\n"
printf "Systemd for kube-controller-manager service found, verifying the authenticity\n"
CLUSTER_SIGNING_CERT_FILE=$(systemctl cat kube-controller-manager.service | grep "\--cluster-signing-cert-file" | awk '{print $1}' | cut -d "=" -f2)
CLUSTER_SIGNING_KEY_FILE=$(systemctl cat kube-controller-manager.service | grep "\--cluster-signing-key-file" | awk '{print $1}' | cut -d "=" -f2)
KUBECONFIG=$(systemctl cat kube-controller-manager.service | grep "\--kubeconfig" | awk '{print $1}' | cut -d "=" -f2)
@@ -531,242 +383,180 @@ check_systemd_kcm()
if [ $CLUSTER_SIGNING_CERT_FILE == $CACERT ] && [ $CLUSTER_SIGNING_KEY_FILE == $CAKEY ] && [ $KUBECONFIG == $KCMKUBECONFIG ] && \
[ $ROOT_CA_FILE == $CACERT ] && [ $SERVICE_ACCOUNT_PRIVATE_KEY_FILE == $SAKEY ]
then
printf "${SUCCESS}kube-controller-manager cluster-signing-cert-file, cluster-signing-key-file, kubeconfig, root-ca-file, service-account-private-key-file are correct\n"
printf "${SUCCESS}kube-controller-manager cluster-signing-cert-file, cluster-signing-key-file, kubeconfig, root-ca-file, service-account-private-key-file are correct\n${NC}"
else
printf "${FAILED}Exiting...Found mismtach in the kube-controller-manager cluster-signing-cert-file, cluster-signing-key-file, kubeconfig, root-ca-file, service-account-private-key-file ,More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/08-bootstrapping-kubernetes-controllers.md#configure-the-kubernetes-controller-manager\n"
printf "${FAILED}Exiting...Found mismtach in the kube-controller-manager cluster-signing-cert-file, cluster-signing-key-file, kubeconfig, root-ca-file, service-account-private-key-file. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/08-bootstrapping-kubernetes-controllers.md#configure-the-kubernetes-controller-manager\n${NC}"
exit 1
fi
else
printf "${FAILED}kube-controller-manager.crt / kube-controller-manager.key is missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/08-bootstrapping-kubernetes-controllers.md#configure-the-kubernetes-controller-manager\n"
printf "${FAILED}kube-controller-manager.crt / kube-controller-manager.key is missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/08-bootstrapping-kubernetes-controllers.md#configure-the-kubernetes-controller-manager\n${NC}"
exit 1
fi
}
check_systemd_ks()
{
KSCERT=/var/lib/kubernetes/kube-scheduler.crt
KSKEY=/var/lib/kubernetes/kube-scheduler.key
KSCERT=/var/lib/kubernetes/pki/kube-scheduler.crt
KSKEY=/var/lib/kubernetes/pki/kube-scheduler.key
KSKUBECONFIG=/var/lib/kubernetes/kube-scheduler.kubeconfig
if [ -z $KSCERT ] && [ -z $KSKEY ]
then
printf "${FAILED}please specify cert and key location\n"
printf "${FAILED}please specify cert and key location\n${NC}"
exit 1
elif [ -f $SYSTEMD_KS_FILE ]
then
printf "${NC}Systemd for kube-scheduler service found, verifying the authenticity\n"
printf "Systemd for kube-scheduler service found, verifying the authenticity\n"
KUBECONFIG=$(systemctl cat kube-scheduler.service | grep "\--kubeconfig"| awk '{print $1}'| cut -d "=" -f2)
ADDRESS=$(systemctl cat kube-scheduler.service | grep "\--address"| awk '{print $1}'| cut -d "=" -f2)
if [ $KUBECONFIG == $KSKUBECONFIG ] && [ $ADDRESS == "127.0.0.1" ]
if [ $KUBECONFIG == $KSKUBECONFIG ]
then
printf "${SUCCESS}kube-scheduler --kubeconfig, --address are correct\n"
printf "${SUCCESS}kube-scheduler --kubeconfig is correct\n${NC}"
else
printf "${FAILED}Exiting...Found mismtach in the kube-scheduler --kubeconfig, --address, More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/08-bootstrapping-kubernetes-controllers.md#configure-the-kubernetes-scheduler\n"
printf "${FAILED}Exiting...Found mismtach in the kube-scheduler --kubeconfig. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/08-bootstrapping-kubernetes-controllers.md#configure-the-kubernetes-scheduler\n${NC}"
exit 1
fi
else
printf "${FAILED}kube-scheduler.crt / kube-scheduler.key is missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/08-bootstrapping-kubernetes-controllers.md#configure-the-kubernetes-scheduler\n"
printf "${FAILED}kube-scheduler.crt / kube-scheduler.key is missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/08-bootstrapping-kubernetes-controllers.md#configure-the-kubernetes-scheduler\n${NC}"
exit 1
fi
}
# END OF Function - Master node #
# Function - Worker-1 node #
check_cert_worker_1()
{
if [ -z $WORKER_1_CERT ] && [ -z $WORKER_1_KEY ]
then
printf "${FAILED}please specify cert and key location of worker-1 node\n"
exit 1
elif [ -f $WORKER_1_CERT ] && [ -f $WORKER_1_KEY ]
then
printf "${NC}worker-1 cert and key found, verifying the authenticity\n"
WORKER_1_CERT_SUBJECT=$(openssl x509 -in $WORKER_1_CERT -text | grep "Subject: CN"| tr -d " ")
WORKER_1_CERT_ISSUER=$(openssl x509 -in $WORKER_1_CERT -text | grep "Issuer: CN"| tr -d " ")
WORKER_1_CERT_MD5=$(openssl x509 -noout -modulus -in $WORKER_1_CERT | openssl md5| awk '{print $2}')
WORKER_1_KEY_MD5=$(openssl rsa -noout -modulus -in $WORKER_1_KEY | openssl md5| awk '{print $2}')
if [ $WORKER_1_CERT_SUBJECT == "Subject:CN=system:node:worker-1,O=system:nodes" ] && [ $WORKER_1_CERT_ISSUER == "Issuer:CN=KUBERNETES-CA" ] && [ $WORKER_1_CERT_MD5 == $WORKER_1_KEY_MD5 ]
then
printf "${SUCCESS}worker-1 cert and key are correct\n"
else
printf "${FAILED}Exiting...Found mismtach in the worker-1 certificate and keys, More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/09-bootstrapping-kubernetes-workers.md#provisioning--kubelet-client-certificates\n"
exit 1
fi
else
printf "${FAILED}/var/lib/kubelet/worker-1.crt / /var/lib/kubelet/worker-1.key is missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/09-bootstrapping-kubernetes-workers.md#provisioning--kubelet-client-certificates\n"
exit 1
fi
}
check_cert_worker_1_kubeconfig()
{
if [ -z $WORKER_1_KUBECONFIG ]
then
printf "${FAILED}please specify worker-1 kubeconfig location\n"
exit 1
elif [ -f $WORKER_1_KUBECONFIG ]
then
printf "${NC}worker-1 kubeconfig file found, verifying the authenticity\n"
WORKER_1_KUBECONFIG_SUBJECT=$(cat $WORKER_1_KUBECONFIG | grep "client-certificate-data:" | awk '{print $2}' | base64 --decode | openssl x509 -text | grep "Subject: CN" | tr -d " ")
WORKER_1_KUBECONFIG_ISSUER=$(cat $WORKER_1_KUBECONFIG | grep "client-certificate-data:" | awk '{print $2}' | base64 --decode | openssl x509 -text | grep "Issuer: CN" | tr -d " ")
WORKER_1_KUBECONFIG_CERT_MD5=$(cat $WORKER_1_KUBECONFIG | grep "client-certificate-data:" | awk '{print $2}' | base64 --decode | openssl x509 -noout | openssl md5 | awk '{print $2}')
WORKER_1_KUBECONFIG_KEY_MD5=$(cat $WORKER_1_KUBECONFIG | grep "client-key-data" | awk '{print $2}' | base64 --decode | openssl rsa -noout | openssl md5 | awk '{print $2}')
WORKER_1_KUBECONFIG_SERVER=$(cat $WORKER_1_KUBECONFIG | grep "server:"| awk '{print $2}')
if [ $WORKER_1_KUBECONFIG_SUBJECT == "Subject:CN=system:node:worker-1,O=system:nodes" ] && [ $WORKER_1_KUBECONFIG_ISSUER == "Issuer:CN=KUBERNETES-CA" ] && \
[ $WORKER_1_KUBECONFIG_CERT_MD5 == $WORKER_1_KUBECONFIG_KEY_MD5 ] && [ $WORKER_1_KUBECONFIG_SERVER == "https://192.168.5.30:6443" ]
then
printf "${SUCCESS}worker-1 kubeconfig cert and key are correct\n"
else
printf "${FAILED}Exiting...Found mismtach in the worker-1 kubeconfig certificate and keys, More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/09-bootstrapping-kubernetes-workers.md#the-kubelet-kubernetes-configuration-file\n"
exit 1
fi
else
printf "${FAILED}worker-1 /var/lib/kubelet/kubeconfig file is missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/09-bootstrapping-kubernetes-workers.md#the-kubelet-kubernetes-configuration-file\n"
exit 1
fi
}
check_cert_worker_1_kubelet()
{
CACERT=/var/lib/kubernetes/ca.crt
WORKER_1_TLSCERTFILE=/var/lib/kubelet/${HOSTNAME}.crt
WORKER_1_TLSPRIVATEKEY=/var/lib/kubelet/${HOSTNAME}.key
if [ -z $WORKER_1_KUBELET ] && [ -z $SYSTEMD_WORKER_1_KUBELET ]
then
printf "${FAILED}please specify worker-1 kubelet config location\n"
exit 1
elif [ -f $WORKER_1_KUBELET ] && [ -f $SYSTEMD_WORKER_1_KUBELET ] && [ -f $WORKER_1_TLSCERTFILE ] && [ -f $WORKER_1_TLSPRIVATEKEY ]
then
printf "${NC}worker-1 kubelet config file, systemd services, tls cert and key found, verifying the authenticity\n"
WORKER_1_KUBELET_CA=$(cat $WORKER_1_KUBELET | grep "clientCAFile:" | awk '{print $2}' | tr -d " \"")
WORKER_1_KUBELET_DNS=$(cat $WORKER_1_KUBELET | grep "resolvConf:" | awk '{print $2}' | tr -d " \"")
WORKER_1_KUBELET_AUTH_MODE=$(cat $WORKER_1_KUBELET | grep "mode:" | awk '{print $2}' | tr -d " \"")
if [ $WORKER_1_KUBELET_CA == $CACERT ] && [ $WORKER_1_KUBELET_DNS == "/run/systemd/resolve/resolv.conf" ] && \
[ $WORKER_1_KUBELET_AUTH_MODE == "Webhook" ]
then
printf "${SUCCESS}worker-1 kubelet config CA cert, resolvConf and Auth mode are correct\n"
else
printf "${FAILED}Exiting...Found mismtach in the worker-1 kubelet config CA cert, resolvConf and Auth mode, More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/09-bootstrapping-kubernetes-workers.md#configure-the-kubelet\n"
exit 1
fi
KUBELETCONFIG=$(systemctl cat kubelet.service | grep "\--config" | awk '{print $1}'| cut -d "=" -f2)
TLSCERTFILE=$(systemctl cat kubelet.service | grep "\--tls-cert-file" | awk '{print $1}'| cut -d "=" -f2)
TLSPRIVATEKEY=$(systemctl cat kubelet.service | grep "\--tls-private-key-file" | awk '{print $1}'| cut -d "=" -f2)
if [ $KUBELETCONFIG == $WORKER_1_KUBELET ] && [ $TLSCERTFILE == $WORKER_1_TLSCERTFILE ] && \
[ $TLSPRIVATEKEY == $WORKER_1_TLSPRIVATEKEY ]
then
printf "${SUCCESS}worker-1 kubelet systemd services are correct\n"
else
printf "${FAILED}Exiting...Found mismtach in the worker-1 kubelet systemd services, More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/09-bootstrapping-kubernetes-workers.md#configure-the-kubelet\n"
exit 1
fi
else
printf "${FAILED}worker-1 kubelet config, systemd services, tls cert and key file is missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/09-bootstrapping-kubernetes-workers.md\n"
exit 1
fi
}
check_cert_worker_1_kp()
{
WORKER_1_KP_CONFIG_YAML=/var/lib/kube-proxy/kube-proxy-config.yaml
if [ -z $WORKER_1_KP_KUBECONFIG ] && [ -z $SYSTEMD_WORKER_1_KP ]
then
printf "${FAILED}please specify worker-1 kube-proxy config and systemd service path\n"
exit 1
elif [ -f $WORKER_1_KP_KUBECONFIG ] && [ -f $SYSTEMD_WORKER_1_KP ] && [ -f $WORKER_1_KP_CONFIG_YAML ]
then
printf "${NC}worker-1 kube-proxy kubeconfig, systemd services and configuration files found, verifying the authenticity\n"
KP_CONFIG=$(cat $WORKER_1_KP_CONFIG_YAML | grep "kubeconfig:" | awk '{print $2}' | tr -d " \"")
KP_CONFIG_YAML=$(systemctl cat kube-proxy.service | grep "\--config" | awk '{print $1}'| cut -d "=" -f2)
if [ $KP_CONFIG == $WORKER_1_KP_KUBECONFIG ] && [ $KP_CONFIG_YAML == $WORKER_1_KP_CONFIG_YAML ]
then
printf "${SUCCESS}worker-1 kube-proxy kubeconfig and configuration files are correct\n"
else
printf "${FAILED}Exiting...Found mismtach in the worker-1 kube-proxy kubeconfig and configuration files, More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/09-bootstrapping-kubernetes-workers.md#configure-the-kubernetes-proxy\n"
exit 1
fi
else
printf "${FAILED}worker-1 kube-proxy kubeconfig and configuration files are missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/09-bootstrapping-kubernetes-workers.md#configure-the-kubernetes-proxy\n"
exit 1
fi
}
# END OF Function - Worker-1 node #
echo -e "This script will validate the certificates in master as well as worker-1 nodes. Before proceeding, make sure you ssh into the respective node [ Master or Worker-1 ] for certificate validation\n"
echo -e "1. Verify certification in Master Node\n"
echo -e "2. Verify certification in Worker-1 Node\n"
echo -e "Please select either the option 1 or 2\n"
echo "This script will validate the certificates in master as well as worker-1 nodes. Before proceeding, make sure you ssh into the respective node [ Master or Worker-1 ] for certificate validation"
echo
echo " 1. Verify certificates on Master Nodes after step 4"
echo " 2. Verify kubeconfigs on Master Nodes after step 5"
echo " 3. Verify kubeconfigs and PKI on Master Nodes after step 8"
echo " 4. Verify kubeconfigs and PKI on worker-1 Node after step 10"
echo " 5. Verify kubeconfigs and PKI on worker-2 Node after step 11"
echo
echo -n "Please select one of the above options: "
read value
HOST=$(hostname -s)
CERT_ISSUER="Issuer:CN=KUBERNETES-CA,O=Kubernetes"
SUBJ_CA="Subject:CN=KUBERNETES-CA,O=Kubernetes"
SUBJ_ADMIN="Subject:CN=admin,O=system:masters"
SUBJ_KCM="Subject:CN=system:kube-controller-manager,O=system:kube-controller-manager"
SUBJ_KP="Subject:CN=system:kube-proxy,O=system:node-proxier"
SUBJ_KS="Subject:CN=system:kube-scheduler,O=system:kube-scheduler"
SUBJ_API="Subject:CN=kube-apiserver,O=Kubernetes"
SUBJ_SA="Subject:CN=service-accounts,O=Kubernetes"
SUBJ_ETCD="Subject:CN=etcd-server,O=Kubernetes"
SUBJ_APIKC="Subject:CN=kube-apiserver-kubelet-client,O=system:masters"
case $value in
1)
if ! [ "${HOST}" = "master-1" -o "${HOST}" = "master-2" ]
then
printf "${FAILED}Must run on master-1 or master-2${NC}\n"
exit 1
fi
echo -e "The selected option is $value, proceeding the certificate verification of Master node"
### MASTER NODES ###
master_hostname=$(hostname -s)
# CRT & KEY verification
check_cert_ca
CERT_LOCATION=$HOME
check_cert_and_key "ca" $SUBJ_CA $CERT_ISSUER
check_cert_and_key "kube-apiserver" $SUBJ_API $CERT_ISSUER
check_cert_and_key "kube-controller-manager" $SUBJ_KCM $CERT_ISSUER
check_cert_and_key "kube-scheduler" $SUBJ_KS $CERT_ISSUER
check_cert_and_key "service-account" $SUBJ_SA $CERT_ISSUER
check_cert_and_key "apiserver-kubelet-client" $SUBJ_APIKC $CERT_ISSUER
check_cert_and_key "etcd-server" $SUBJ_ETCD $CERT_ISSUER
if [ $master_hostname == "master-1" ]
then
check_cert_admin
check_cert_kcm
check_cert_kp
check_cert_ks
check_cert_adminkubeconfig
check_cert_kpkubeconfig
if [ "${HOST}" = "master-1" ]
then
check_cert_and_key "admin" $SUBJ_ADMIN $CERT_ISSUER
check_cert_and_key "kube-proxy" $SUBJ_KP $CERT_ISSUER
fi
check_cert_api
check_cert_sa
check_cert_etcd
# Kubeconfig verification
check_cert_kcmkubeconfig
check_cert_kskubeconfig
# Systemd verification
check_systemd_etcd
check_systemd_api
check_systemd_kcm
check_systemd_ks
### END OF MASTER NODES ###
;;
2)
echo -e "The selected option is $value, proceeding the certificate verification of Worker-1 node"
if ! [ "${HOST}" = "master-1" -o "${HOST}" = "master-2" ]
then
printf "${FAILED}Must run on master-1 or master-2${NC}\n"
exit 1
fi
### WORKER-1 NODE ###
check_cert_adminkubeconfig
check_kubeconfig_exists "kube-controller-manager" $HOME
check_kubeconfig_exists "kube-scheduler" $HOME
check_cert_worker_1
check_cert_worker_1_kubeconfig
check_cert_worker_1_kubelet
check_cert_worker_1_kp
### END OF WORKER-1 NODE ###
if [ "${HOST}" = "master-1" ]
then
check_kubeconfig_exists "kube-proxy" $HOME
fi
;;
3)
if ! [ "${HOST}" = "master-1" -o "${HOST}" = "master-2" ]
then
printf "${FAILED}Must run on master-1 or master-2${NC}\n"
exit 1
fi
CERT_LOCATION=/etc/etcd
check_cert_only "ca" $SUBJ_CA $CERT_ISSUER
check_cert_and_key "etcd-server" $SUBJ_ETCD $CERT_ISSUER
CERT_LOCATION=/var/lib/kubernetes/pki
check_cert_and_key "ca" $SUBJ_CA $CERT_ISSUER
check_cert_and_key "kube-apiserver" $SUBJ_API $CERT_ISSUER
check_cert_and_key "kube-controller-manager" $SUBJ_KCM $CERT_ISSUER
check_cert_and_key "kube-scheduler" $SUBJ_KS $CERT_ISSUER
check_cert_and_key "service-account" $SUBJ_SA $CERT_ISSUER
check_cert_and_key "apiserver-kubelet-client" $SUBJ_APIKC $CERT_ISSUER
check_cert_and_key "etcd-server" $SUBJ_ETCD $CERT_ISSUER
check_kubeconfig "kube-controller-manager" "/var/lib/kubernetes" "https://127.0.0.1:6443"
check_kubeconfig "kube-scheduler" "/var/lib/kubernetes" "https://127.0.0.1:6443"
check_systemd_api
check_systemd_etcd
check_systemd_kcm
check_systemd_ks
;;
4)
if ! [ "${HOST}" = "worker-1" ]
then
printf "${FAILED}Must run on worker-1${NC}\n"
exit 1
fi
CERT_LOCATION=/var/lib/kubernetes/pki
check_cert_only "ca" $SUBJ_CA $CERT_ISSUER
check_cert_and_key "kube-proxy" $SUBJ_KP $CERT_ISSUER
check_cert_and_key "worker-1" "Subject:CN=system:node:worker-1,O=system:nodes" $CERT_ISSUER
check_kubeconfig "kube-proxy" "/var/lib/kube-proxy" "https://${LOADBALANCER}:6443"
check_kubeconfig "kubelet" "/var/lib/kubelet" "https://${LOADBALANCER}:6443"
;;
5)
if ! [ "${HOST}" = "worker-2" ]
then
printf "${FAILED}Must run on worker-2${NC}\n"
exit 1
fi
CERT_LOCATION=/var/lib/kubernetes/pki
check_cert_only "ca" $SUBJ_CA $CERT_ISSUER
check_cert_and_key "kube-proxy" $SUBJ_KP $CERT_ISSUER
CERT_LOCATION=/var/lib/kubelet/pki
check_cert_only "kubelet-client-current" "Subject:O=system:nodes,CN=system:node:worker-2" $CERT_ISSUER
check_kubeconfig "kube-proxy" "/var/lib/kube-proxy" "https://${LOADBALANCER}:6443"
;;
*)
printf "${FAILED}Exiting.... Please select the valid option either 1 or 2\n"
printf "${FAILED}Exiting.... Please select the valid option either 1 or 2\n${NC}"
exit 1
;;
esac

View File

@@ -1,3 +0,0 @@
cd /tmp
curl -fsSL https://get.docker.com -o get-docker.sh
sh /tmp/get-docker.sh

View File

@@ -1,15 +0,0 @@
#!/bin/bash
export DEBIAN_FRONTEND=noninteractive
apt-get update \
&& apt-get install -y \
apt-transport-https \
ca-certificates \
curl \
software-properties-common \
&& curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add - \
&& add-apt-repository \
"deb https://download.docker.com/linux/$(. /etc/os-release; echo "$ID") \
$(lsb_release -cs) \
stable" \
&& apt-get update \
&& apt-get install -y docker-ce=$(apt-cache madison docker-ce | grep 18.06 | head -1 | awk '{print $3}')

View File

@@ -0,0 +1,19 @@
#!/bin/bash
#
# Sets up the kernel with the requirements for running Kubernetes
# Requires a reboot, which is carried out by the vagrant provisioner.
set -ex
# Disable cgroups v2 (kernel command line parameter)
sed -i 's/GRUB_CMDLINE_LINUX_DEFAULT="/GRUB_CMDLINE_LINUX_DEFAULT="systemd.unified_cgroup_hierarchy=0 ipv6.disable=1 /' /etc/default/grub
update-grub
# Add br_netfilter kernel module
echo "br_netfilter" >> /etc/modules
# Set network tunables
cat <<EOF >> /etc/sysctl.d/10-kubernetes.conf
net.bridge.bridge-nf-call-iptables=1
net.ipv4.ip_forward=1
EOF

3
vagrant/ubuntu/tmux.conf Normal file
View File

@@ -0,0 +1,3 @@
set -g default-shell /bin/bash
set -g mouse on
bind -n C-x setw synchronize-panes

View File

@@ -1,5 +1,6 @@
#!/bin/bash
# Point to Google's DNS server
sed -i -e 's/#DNS=/DNS=8.8.8.8/' /etc/systemd/resolved.conf
service systemd-resolved restart

View File

@@ -1,17 +1,22 @@
#!/bin/bash
set -e
#
# Set up /etc/hosts so we can resolve all the machines in the VirtualBox network
set -ex
IFNAME=$1
THISHOST=$2
ADDRESS="$(ip -4 addr show $IFNAME | grep "inet" | head -1 |awk '{print $2}' | cut -d/ -f1)"
NETWORK=$(echo $ADDRESS | awk 'BEGIN {FS="."} ; { printf("%s.%s.%s", $1, $2, $3) }')
sed -e "s/^.*${HOSTNAME}.*/${ADDRESS} ${HOSTNAME} ${HOSTNAME}.local/" -i /etc/hosts
# remove ubuntu-bionic entry
sed -e '/^.*ubuntu-bionic.*/d' -i /etc/hosts
# remove ubuntu-jammy entry
sed -e '/^.*ubuntu-jammy.*/d' -i /etc/hosts
sed -e "/^.*$2.*/d" -i /etc/hosts
# Update /etc/hosts about other hosts
cat >> /etc/hosts <<EOF
192.168.5.11 master-1
192.168.5.12 master-2
192.168.5.21 worker-1
192.168.5.22 worker-2
192.168.5.30 lb
${NETWORK}.11 master-1
${NETWORK}.12 master-2
${NETWORK}.21 worker-1
${NETWORK}.22 worker-2
${NETWORK}.30 loadbalancer
EOF

6
vagrant/ubuntu/vimrc Normal file
View File

@@ -0,0 +1,6 @@
set nu
set ts=2
set sw=2
set et
set ai
set pastetoggle=<F3>