feat: Added Virtual Machines
This commit introduces a new directory structure for managing virtual machines, specifically tailored for setting up Kubernetes manually on Ubuntu. Files have been added to the `virtual-machines` directory, which includes scripts and configurations necessary for the setup. The following files have been added: - `README.md`: Documentation for the virtual machines setup. - `Vagrantfile`: Configuration for Vagrant to manage the virtual machines. - `ssh.sh`: Script to set up SSH access. - `tmux.conf`: Configuration file for tmux. - `update-dns.sh`: Script to update DNS settings. - `controlplane.sh`: Script to set up the control plane. - `setup-hosts.sh`: Script to configure the hosts file. - `vimrc`: Configuration file for Vim. This commit is part of the ongoing effort to enhance the documentation and setup instructions for manual installation of Kubernetes. Making it easier for users to get started with manual installations and configurations. With the intended purpose of making them more capabale administrators and users of Kubernetes.pull/881/head
Khalifah Shabazz
parent
52eb26dad1
commit
4bd744480a
|
|
@ -0,0 +1,108 @@
|
|||
# Prerequisites
|
||||
|
||||
If you have an M-series (Apple Silicon) Mac, you cannot run VirtualBox. Please instead see our [Apple Silicon](../../apple-silicon/) guide.
|
||||
|
||||
## VM Hardware Requirements
|
||||
|
||||
* 8 GB of RAM (16 preferred)
|
||||
* 8-core/4-core hyper threaded or better CPU, e.g. Core-i7/Core-i9 (will be slow otherwise)
|
||||
* 50 GB Disk space
|
||||
|
||||
## git
|
||||
|
||||
Required to download the repo. It is normally pre-installed on Mac, but not on Windows. If you need to install it, see [here](https://git-scm.com/download).
|
||||
|
||||
## VirtualBox
|
||||
|
||||
Download and Install [VirtualBox](https://www.virtualbox.org/wiki/Downloads) on any one of the supported platforms:
|
||||
|
||||
- Windows hosts
|
||||
- MacOS hosts x86 only. For Apple Silicon (M-series processors), see [here](../../apple-silicon/).
|
||||
- Linux distributions
|
||||
|
||||
This lab was last tested with VirtualBox 7.0.12, though newer versions should be ok.
|
||||
|
||||
## Vagrant
|
||||
|
||||
Once VirtualBox is installed you may choose to deploy virtual machines manually on it.
|
||||
Vagrant provides an easier way to deploy multiple virtual machines on VirtualBox more consistently.
|
||||
|
||||
Download and Install [Vagrant](https://www.vagrantup.com/) on your platform.
|
||||
|
||||
This lab was last tested with Vagrant 2.3.7, though newer versions should be ok.
|
||||
|
||||
## Lab Defaults
|
||||
|
||||
The labs have been configured with the following networking defaults. If you change any of these after you have deployed any of the lab, you'll need to completely reset it and start again from the beginning:
|
||||
|
||||
```bash
|
||||
vagrant destroy -f
|
||||
vagrant up
|
||||
```
|
||||
|
||||
If you do change any of these, **please consider that a personal preference and don't submit a PR for it**.
|
||||
|
||||
### Virtual Machine Network
|
||||
|
||||
Due to how VirtualBox/Vagrant works, the networking for each VM requires two network adapters; one NAT (`enp0s3`) to communicate with the outside world, and one internal (`enp0s8`) which is attached to the VirtualBox network mentioned above. By default, Kubernetes components will connect to the default network adapter - the NAT one, which is *not* what we want, therefore we have pre-set an environment variable `PRIMARY_IP` on all VMs which is the IP address that Kubernetes components should be using. In the coming labs you will see this environment variable being used to ensure Kubernetes components bind to the correct network interface.
|
||||
|
||||
`PRIMARY_IP` is defined as the IP address of the network interface on the node that is connected to the network having the default gateway, and is the interface that a node will use to talk to the other nodes. For those interested, this variable is assigned the result of the following command
|
||||
|
||||
```bash
|
||||
ip route | grep default | awk '{ print $9 }'
|
||||
```
|
||||
|
||||
#### Bridge Networking
|
||||
|
||||
The default configuration in this lab is to bring the VMs up on bridged interfaces. What this means is that your Kubernetes nodes will appear as additional machines on your local network, their IP addresses being provided dynamically by your broadband router. This facilitates the use of your browser to connect to any NodePort services you deploy.
|
||||
|
||||
Should you have issues deploying bridge networking, please raise a [bug report](https://github.com/kodekloudhub/certified-kubernetes-administrator-course/issues) and include all details including the output of `vagrant up`.
|
||||
|
||||
Then retry the lab in NAT mode. How to do this is covered in the [next section](./02-compute-resources.md).
|
||||
|
||||
#### NAT Networking
|
||||
|
||||
In NAT configuration, the network on which the VMs run is isolated from your broadband router's network by a NAT gateway managed by the hypervisor. This means that VMs can see out (and connect to Internet), but you can't see in (i.e. use browser to connect to NodePorts) without setting up individual port forwarding rules for every NodePort using the VirtualBox UI.
|
||||
|
||||
The network used by the Virtual Box virtual machines is `192.168.56.0/24`.
|
||||
|
||||
To change this, edit the [Vagrantfile](../Vagrantfile) in your cloned copy (do not edit directly in github), and set the new value for the network prefix at line 9. This should not overlap any of the other network settings.
|
||||
|
||||
Note that you do not need to edit any of the other scripts to make the above change. It is all managed by shell variable computations based on the assigned VM IP addresses and the values in the hosts file (also computed).
|
||||
|
||||
It is *recommended* that you leave the pod and service networks as the defaults. If you change them then you will also need to edit the Weave networking manifests to accommodate your change.
|
||||
|
||||
If you do decide to change any of these, please treat as personal preference and do not raise a pull request.
|
||||
|
||||
|
||||
### Pod Network
|
||||
|
||||
The network used to assign IP addresses to pods is `10.244.0.0/16`.
|
||||
|
||||
To change this, open all the `.md` files in the [docs](../docs/) directory in your favourite IDE and do a global replace on<br>
|
||||
`POD_CIDR=10.244.0.0/16`<br>
|
||||
with the new CIDR range. This should not overlap any of the other network settings.
|
||||
|
||||
### Service Network
|
||||
|
||||
The network used to assign IP addresses to Cluster IP services is `10.96.0.0/16`.
|
||||
|
||||
To change this, open all the `.md` files in the [docs](../docs/) directory in your favourite IDE and do a global replace on<br>
|
||||
`SERVICE_CIDR=10.96.0.0/16`<br>
|
||||
with the new CIDR range. This should not overlap any of the other network settings.
|
||||
|
||||
## Running Commands in Parallel with tmux
|
||||
|
||||
If you are running this tutorial on an x86 Mac, you can instead use iterm2 to achive this. See the iterm2 setup in the [Apple Silicon guide](../../apple-silicon/docs/01-prerequisites.md#running-commands-in-parallel-with-iterm2).
|
||||
|
||||
[tmux](https://github.com/tmux/tmux/wiki) can be used to run the same commands on multiple compute instances at the same time. Labs in this tutorial may require running the same commands across multiple compute instances. In those cases you may consider using tmux and splitting a window into multiple panes with synchronize-panes enabled to speed up the provisioning process.
|
||||
|
||||
In order to use tmux, you must first connect to `controlplane` and run tmux there. From inside the tmux session you can open multiple panes and ssh to the worker nodes from these panes.
|
||||
|
||||
*The use of tmux is optional and not required to complete this tutorial*.
|
||||
|
||||
|
||||
|
||||
> Enable synchronize-panes by pressing `CTRL+B` followed by `"` to split the window into two panes. In each pane (selectable with mouse), ssh to the host(s) you will be working with.</br>Next type `CTRL+X` at the prompt to begin sync. In sync mode, the dividing line between panes will be red. Everything you type or paste in one pane will be echoed in the other.<br>To disable synchronization type `CTRL+X` again.</br></br>Note that the `CTRL-X` key binding is provided by a `.tmux.conf` loaded onto the VM by the vagrant provisioner.<br/>To paste commands into a tmux pane, use `SHIFT-RightMouseButton`.
|
||||
|
||||
Next: [Compute Resources](02-compute-resources.md)
|
||||
|
|
@ -0,0 +1,254 @@
|
|||
# -*- mode: ruby -*-
|
||||
# vi:set ft=ruby sw=2 ts=2 sts=2:
|
||||
|
||||
# Set the Kubernetes install mode
|
||||
# "MANUAL" - This mode adds a jumpbox and leave some networking undone. Its
|
||||
# meant to be used with a guide like "Kubernetes The Hard Way."
|
||||
# "KUBEADM" This mode is similar to the exam,
|
||||
# 1. Setup /etc/hosts file on each node transitively.
|
||||
# where where you'll be given
|
||||
# machines that have there hostnames and SSH configured, so you won't
|
||||
# need to know so much networking setup. Also no jumpbox is included.
|
||||
INSTALL_MODE = "MANUAL"
|
||||
|
||||
BOOT_TIMEOUT_SEC = 120
|
||||
|
||||
# Set the build mode
|
||||
# "BRIDGE" - Places VMs on your local network so cluster can be accessed from browser.
|
||||
# You must have enough spare IPs on your network for the cluster nodes.
|
||||
# "NAT" - Places VMs in a private virtual network. Cluster cannot be accessed
|
||||
# without setting up a port forwarding rule for every NodePort exposed.
|
||||
# Use this mode if for some reason BRIDGE doesn't work for you.
|
||||
BUILD_MODE = "NAT"
|
||||
|
||||
# Define the number of worker nodes
|
||||
# If this number is changed, remember to update setup-hosts.sh script with the new hosts IP details in /etc/hosts of each VM.
|
||||
NUM_WORKER_NODES = 2
|
||||
|
||||
# Network parameters for NAT mode
|
||||
NAT_IP_PREFIX = "192.168.56"
|
||||
JUMPER_IP_START = 10
|
||||
CONTROLPLANE_NAT_IP = 11
|
||||
NODE_IP_START = 20
|
||||
|
||||
# Host operating system detection
|
||||
module OS
|
||||
def OS.windows?
|
||||
(/cygwin|mswin|mingw|bccwin|wince|emx/ =~ RUBY_PLATFORM) != nil
|
||||
end
|
||||
|
||||
def OS.mac?
|
||||
(/darwin/ =~ RUBY_PLATFORM) != nil
|
||||
end
|
||||
|
||||
def OS.unix?
|
||||
!OS.windows?
|
||||
end
|
||||
|
||||
def OS.linux?
|
||||
OS.unix? and not OS.mac?
|
||||
end
|
||||
|
||||
def OS.jruby?
|
||||
RUBY_ENGINE == "jruby"
|
||||
end
|
||||
end
|
||||
|
||||
# Determine host adapter for bridging in BRIDGE mode
|
||||
def get_bridge_adapter()
|
||||
if OS.windows?
|
||||
return %x{powershell -Command "Get-NetRoute -DestinationPrefix 0.0.0.0/0 | Get-NetAdapter | Select-Object -ExpandProperty InterfaceDescription"}.chomp
|
||||
elsif OS.linux?
|
||||
return %x{ip route | grep default | awk '{ print $5 }'}.chomp
|
||||
elsif OS.mac?
|
||||
return %x{mac/mac-bridge.sh}.chomp
|
||||
end
|
||||
end
|
||||
|
||||
# Helper method to get the machine ID of a node.
|
||||
# This will only be present if the node has been
|
||||
# created in VirtualBox.
|
||||
def get_machine_id(vm_name)
|
||||
machine_id_filepath = ".vagrant/machines/#{vm_name}/virtualbox/id"
|
||||
if not File.exist? machine_id_filepath
|
||||
return nil
|
||||
else
|
||||
return File.read(machine_id_filepath)
|
||||
end
|
||||
end
|
||||
|
||||
# Helper method to determine whether all nodes are up
|
||||
def all_nodes_up()
|
||||
if get_machine_id("controlplane").nil?
|
||||
return false
|
||||
end
|
||||
|
||||
(1..NUM_WORKER_NODES).each do |i|
|
||||
if get_machine_id("node0#{i}").nil?
|
||||
return false
|
||||
end
|
||||
end
|
||||
|
||||
if get_machine_id("jumpbox").nil?
|
||||
return false
|
||||
end
|
||||
|
||||
return true
|
||||
end
|
||||
|
||||
# Sets up hosts file and DNS
|
||||
def setup_dns(node)
|
||||
if INSTALL_MODE == "KUBEADM"
|
||||
# Set up /etc/hosts
|
||||
node.vm.provision "setup-hosts", :type => "shell", :path => "ubuntu/vagrant/setup-hosts.sh" do |s|
|
||||
s.args = [NAT_IP_PREFIX, BUILD_MODE, NUM_WORKER_NODES, CONTROLPLANE_NAT_IP, NODE_IP_START]
|
||||
end
|
||||
end
|
||||
# Set up DNS resolution
|
||||
node.vm.provision "setup-dns", type: "shell", :path => "ubuntu/update-dns.sh"
|
||||
end
|
||||
|
||||
# Runs provisioning steps that are required by masters and workers
|
||||
def provision_kubernetes_node(node)
|
||||
# Set up DNS
|
||||
setup_dns node
|
||||
# Set up ssh
|
||||
node.vm.provision "setup-ssh", :type => "shell", :path => "ubuntu/ssh.sh"
|
||||
end
|
||||
|
||||
# All Vagrant configuration is done below. The "2" in Vagrant.configure
|
||||
# configures the configuration version (we support older styles for
|
||||
# backwards compatibility). Please don't change it unless you know what
|
||||
# you're doing.
|
||||
Vagrant.configure("2") do |config|
|
||||
# The most common configuration options are documented and commented below.
|
||||
# For a complete reference, please see the online documentation at
|
||||
# https://docs.vagrantup.com.
|
||||
|
||||
# Every Vagrant development environment requires a box. You can search for
|
||||
# boxes at https://portal.cloud.hashicorp.com/vagrant/discover
|
||||
# config.vm.box = "base"
|
||||
|
||||
config.vm.box = "ubuntu/jammy64"
|
||||
config.vm.boot_timeout = BOOT_TIMEOUT_SEC
|
||||
|
||||
# Set SSH login user and password
|
||||
config.ssh.username = "root"
|
||||
config.ssh.password = "vagrant"
|
||||
|
||||
# Disable automatic box update checking. If you disable this, then
|
||||
# boxes will only be checked for updates when the user runs
|
||||
# `vagrant box outdated`. This is not recommended.
|
||||
config.vm.box_check_update = false
|
||||
|
||||
# Provision controlplane Nodes
|
||||
config.vm.define "controlplane" do |node|
|
||||
# Name shown in the GUI
|
||||
node.vm.provider "virtualbox" do |vb|
|
||||
vb.name = "controlplane"
|
||||
vb.memory = 2048
|
||||
vb.cpus = 2
|
||||
end
|
||||
node.vm.hostname = "controlplane"
|
||||
if BUILD_MODE == "BRIDGE"
|
||||
adapter = ""
|
||||
node.vm.network :public_network, bridge: get_bridge_adapter()
|
||||
else
|
||||
node.vm.network :private_network, ip: NAT_IP_PREFIX + ".#{CONTROLPLANE_NAT_IP}"
|
||||
node.vm.network "forwarded_port", guest: 22, host: "#{2710}"
|
||||
end
|
||||
provision_kubernetes_node node
|
||||
# Install (opinionated) configs for vim and tmux on master-1. These used by the author for CKA exam.
|
||||
node.vm.provision "file", source: "./ubuntu/tmux.conf", destination: "$HOME/.tmux.conf"
|
||||
node.vm.provision "file", source: "./ubuntu/vimrc", destination: "$HOME/.vimrc"
|
||||
end
|
||||
|
||||
# Provision Worker Nodes
|
||||
(1..NUM_WORKER_NODES).each do |i|
|
||||
config.vm.define "node0#{i}" do |node|
|
||||
node.vm.provider "virtualbox" do |vb|
|
||||
vb.name = "node0#{i}"
|
||||
vb.memory = 1024
|
||||
vb.cpus = 1
|
||||
end
|
||||
node.vm.hostname = "node0#{i}"
|
||||
if BUILD_MODE == "BRIDGE"
|
||||
node.vm.network :public_network, bridge: get_bridge_adapter()
|
||||
else
|
||||
node.vm.network :private_network, ip: NAT_IP_PREFIX + ".#{NODE_IP_START + i}"
|
||||
node.vm.network "forwarded_port", guest: 22, host: "#{2720 + i}"
|
||||
end
|
||||
provision_kubernetes_node node
|
||||
end
|
||||
end
|
||||
|
||||
if INSTALL_MODE == "MANUAL"
|
||||
# Provision a JumpBox
|
||||
config.vm.define "jumpbox" do |node|
|
||||
# Name shown in the GUI
|
||||
node.vm.provider "virtualbox" do |vb|
|
||||
vb.name = "jumpbox"
|
||||
vb.memory = 512
|
||||
vb.cpus = 1
|
||||
end
|
||||
node.vm.hostname = "jumpbox"
|
||||
if BUILD_MODE == "BRIDGE"
|
||||
adapter = ""
|
||||
node.vm.network :public_network, bridge: get_bridge_adapter()
|
||||
else
|
||||
node.vm.network :private_network, ip: NAT_IP_PREFIX + ".#{JUMPER_IP_START}"
|
||||
node.vm.network "forwarded_port", guest: 22, host: "#{2730}"
|
||||
end
|
||||
provision_kubernetes_node node
|
||||
end
|
||||
end
|
||||
|
||||
if BUILD_MODE == "BRIDGE"
|
||||
# Trigger that fires after each VM starts.
|
||||
# Does nothing until all the VMs have started, at which point it
|
||||
# gathers the IP addresses assigned to the bridge interfaces by DHCP
|
||||
# and pushes a hosts file to each node with these IPs.
|
||||
config.trigger.after :up do |trigger|
|
||||
trigger.name = "Post provisioner"
|
||||
trigger.ignore = [:destroy, :halt]
|
||||
trigger.ruby do |env, machine|
|
||||
if all_nodes_up()
|
||||
puts " Gathering IP addresses of nodes..."
|
||||
nodes = ["controlplane"]
|
||||
ips = []
|
||||
(1..NUM_WORKER_NODES).each do |i|
|
||||
nodes.push("node0#{i}")
|
||||
end
|
||||
nodes.each do |n|
|
||||
ips.push(%x{vagrant ssh #{n} -c 'public-ip'}.chomp)
|
||||
end
|
||||
hosts = ""
|
||||
ips.each_with_index do |ip, i|
|
||||
hosts << ip << " " << nodes[i] << "\n"
|
||||
end
|
||||
puts " Setting /etc/hosts on nodes..."
|
||||
File.open("hosts.tmp", "w") { |file| file.write(hosts) }
|
||||
nodes.each do |node|
|
||||
system("vagrant upload hosts.tmp /tmp/hosts.tmp #{node}")
|
||||
system("vagrant ssh #{node} -c 'cat /tmp/hosts.tmp | sudo tee -a /etc/hosts'")
|
||||
end
|
||||
File.delete("hosts.tmp")
|
||||
puts <<~EOF
|
||||
|
||||
VM build complete!
|
||||
|
||||
Use either of the following to access any NodePort services you create from your browser
|
||||
replacing "port_number" with the number of your NodePort.
|
||||
|
||||
EOF
|
||||
(1..NUM_WORKER_NODES).each do |i|
|
||||
puts " http://#{ips[i]}:port_number"
|
||||
end
|
||||
puts ""
|
||||
else
|
||||
puts " Nothing to do here"
|
||||
end
|
||||
end
|
||||
end
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
#!/bin/bash
|
||||
|
||||
# Enable password auth in sshd so we can use ssh-copy-id
|
||||
sed -i 's/#PasswordAuthentication/PasswordAuthentication/' /etc/ssh/sshd_config
|
||||
sed -i 's/KbdInteractiveAuthentication no/KbdInteractiveAuthentication yes/' /etc/ssh/sshd_config
|
||||
systemctl restart sshd
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
set -g default-shell /usr/bin/bash
|
||||
set -g mouse on
|
||||
bind -n C-x setw synchronize-panes
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
#!/bin/bash
|
||||
|
||||
# Point to Google's DNS server
|
||||
sed -i -e 's/#DNS=/DNS=8.8.8.8/' /etc/systemd/resolved.conf
|
||||
|
||||
service systemd-resolved restart
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
{
|
||||
POD_CIDR=10.244.0.0/16
|
||||
SERVICE_CIDR=10.96.0.0/16
|
||||
|
||||
kubeadm init --pod-network-cidr $POD_CIDR --service-cidr $SERVICE_CIDR --apiserver-advertise-address $PRIMARY_IP
|
||||
|
||||
kubectl --kubeconfig /etc/kubernetes/admin.conf \
|
||||
apply -f "https://github.com/weaveworks/weave/releases/download/v2.8.1/weave-daemonset-k8s-1.11.yaml"
|
||||
|
||||
}
|
||||
|
|
@ -0,0 +1,52 @@
|
|||
#!/usr/bin/env bash
|
||||
#
|
||||
# Set up /etc/hosts so we can resolve all the nodes
|
||||
|
||||
set -e
|
||||
|
||||
IP_NW=$1
|
||||
BUILD_MODE=$2
|
||||
NUM_WORKER_NODES=$3
|
||||
MASTER_IP_START=$4
|
||||
NODE_IP_START=$5
|
||||
|
||||
if [ "$BUILD_MODE" = "BRIDGE" ]
|
||||
then
|
||||
# Determine machine IP from route table -
|
||||
# Interface that routes to default GW that isn't on the NAT network.
|
||||
sleep 5
|
||||
MY_IP="$(ip route | grep default | grep -Pv '10\.\d+\.\d+\.\d+' | awk '{ print $9 }')"
|
||||
|
||||
# From this, determine the network (which for average broadband we assume is a /24)
|
||||
MY_NETWORK=$(echo $MY_IP | awk 'BEGIN {FS="."} ; { printf("%s.%s.%s", $1, $2, $3) }')
|
||||
|
||||
# Create a script that will return this machine's IP to the bridge post-provisioner.
|
||||
cat <<EOF > /usr/local/bin/public-ip
|
||||
#!/usr/bin/env sh
|
||||
echo -n $MY_IP
|
||||
EOF
|
||||
chmod +x /usr/local/bin/public-ip
|
||||
else
|
||||
# Determine machine IP from route table -
|
||||
# Interface that is connected to the NAT network.
|
||||
MY_IP="$(ip route | grep "^$IP_NW" | awk '{print $NF}')"
|
||||
MY_NETWORK=$IP_NW
|
||||
fi
|
||||
|
||||
# Remove unwanted entries
|
||||
sed -e '/^.*ubuntu-jammy.*/d' -i /etc/hosts
|
||||
sed -e "/^.*${HOSTNAME}.*/d" -i /etc/hosts
|
||||
|
||||
# Export PRIMARY IP as an environment variable
|
||||
echo "PRIMARY_IP=${MY_IP}" >> /etc/environment
|
||||
|
||||
[ "$BUILD_MODE" = "BRIDGE" ] && exit 0
|
||||
|
||||
# Update /etc/hosts about other hosts (NAT mode)
|
||||
echo "${MY_NETWORK}.${MASTER_IP_START} controlplane" >> /etc//hosts
|
||||
for i in $(seq 1 $NUM_WORKER_NODES)
|
||||
do
|
||||
num=$(( $NODE_IP_START + $i ))
|
||||
echo "${MY_NETWORK}.${num} node0${i}" >> /etc//hosts
|
||||
done
|
||||
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
set nu
|
||||
set ts=2
|
||||
set sw=2
|
||||
set et
|
||||
set ai
|
||||
set pastetoggle=<F3>
|
||||
Loading…
Reference in New Issue