kubernetes-the-hard-way/docs/01-infrastructure-azure.md

# Cloud Infrastructure Provisioning - Azure
This lab will walk you through provisioning the compute instances required for running a H/A Kubernetes cluster. A total of 10 virtual machines will be created.

The guide assumes you'll be creating resources in the `West Us` region as a single Azure Resource Manager resource group.


> All machines will be provisioned with fixed private IP addresses to simplify the bootstrap process.

The cluster VMs are only accessible via a jump box (a VM with publicly accessible ssh endpoint). The workers machines are exposed via external load balancer that carries both an public IP and public FQDN. 


## Variables

Change the following values as needed. 

```
# dns for jumpbox is <jumpboxDnsLabel>.westus.cloudapp.azure.com
jumpboxDnsLabel="the-hard-way-jumpbox" 
```

```
# dns for workers is <workersDnsLabel>.westus.cloudapp.azure.com
workersDnsLabel="the-hard-way" 
```

```
#storage account used by jumpbox + controllers + Etcd VMs
controlPlaneStorageAccount="thehardwaycsa"
```

```
#storage account used by workers VMs
workersStorageAccount="thehardwaywsa"
```

```
# all vms are using ubunut 16.4 LTS 
imageUrn="Canonical:UbuntuServer:16.04.0-LTS:latest"
```

## Create Resource Group

```
azure group create \
	--name the-hard-way \
	--location "West Us"
```

## Networking

### Create Routing Table

```
azure network route-table create \
	--resource-group the-hard-way \
	--name the-hard-way-rtable \
	--location "West Us"
```

### Create Network Security Group (NSG)

```
azure network nsg create \
	--resource-group the-hard-way \
	--name the-hard-way-nsg \
	--location "West Us"
```


Create NSG rule allowing SSH to the jumpbox

```
azure network nsg rule create \
	--resource-group the-hard-way \
	--nsg-name the-hard-way-nsg \
	--name allow-ssh-jumpbox \
	--protocol tcp \
	--access allow  \
	--destination-address-prefix 10.0.0.5 \
	--destination-port-range 22 \
	--priority 100 \
	--direction inbound
```


### Create VNET + Subnet 

Cluster VNET
```
azure network vnet create \
	--resource-group the-hard-way \
	--name the-hard-way-net \
	--address-prefixes 10.0.0.0/8 \
	--location "West Us"
```

Create subnets

```
# Azure UDR "user defined routes" in custom routing tables 
# routes traffic leaving the subnet.
# Workers & pods (IPs) have to be in two separate subnets

azure network vnet subnet create \
	--resource-group the-hard-way \
	--vnet-name the-hard-way-net \
	--name kubernetes-mgmt \
	--address-prefix 10.0.0.0/16


azure network vnet subnet create \
	--resource-group the-hard-way \
	--vnet-name the-hard-way-net \
	--name kubernetes \
	--address-prefix 10.224.0.0/11


```

Associate the routing table and NSG to Kubernetes/-mgmt subnets

```
azure network vnet subnet set \
	--resource-group the-hard-way \
	--vnet-name the-hard-way-net \
	--name kubernetes-mgmt \
	--network-security-group-name the-hard-way-nsg \
	--route-table-name the-hard-way-rtable


azure network vnet subnet set \
	--resource-group the-hard-way \
	--vnet-name the-hard-way-net \
	--name kubernetes \
	--network-security-group-name the-hard-way-nsg \
	--route-table-name the-hard-way-rtable
```


Create public IP + DNS label for the jumpbox

```
azure network public-ip create \
	--resource-group the-hard-way \
	--name the-hard-way-jumpbox  \
	--allocation-method Dynamic \
	--domain-name-label $jumpboxDnsLabel \
	--location "West Us"
```

## Virtual Machines

Create SSH keys (Used by All VMs)

```
mkdir keys
ssh-keygen -t rsa -f ./keys/cluster
```

### Storage Accounts 

Create storage account for control plane VMs (Etcd & Controllers)

```
azure storage account create $controlPlaneStorageAccount \
	--resource-group the-hard-way \
	--kind storage \
	--sku-name LRS \
	--location "West Us"
```

Create storage account for workers VMs

```
azure storage account create $workersStorageAccount \
	--resource-group the-hard-way \
	--kind storage \
	--sku-name LRS \
	--location "West Us"
```



### Jump Box

#### Create NIC (Private IP + Public IP + FQDN)

```
azure network nic create \
	--resource-group the-hard-way \
	--name jumpbox-nic \
	--private-ip-address "10.0.0.5" \
	--subnet-vnet-name the-hard-way-net  \
	--subnet-name kubernetes-mgmt  \
	--public-ip-name the-hard-way-jumpbox \
	--location "West Us"
```

#### Create VM

```
azure vm create \
	--resource-group the-hard-way \
	--name jumpbox \
	--vm-size Standard_A1 \
	--nic-name jumpbox-nic \
	--vnet-name the-hard-way-net  \
	--vnet-subnet-name kubernetes-mgmt  \
	--os-type linux \
	--image-urn $imageUrn \
	--storage-account-name $controlPlaneStorageAccount \
	--storage-account-container-name vhds \
	--os-disk-vhd jumpbox.vhd \
	--admin-username thehardway \
	--ssh-publickey-file ./keys/cluster.pub \
	--location "West US"
``` 

### Etcd 

#### Etcd 0

Create Nic
```
azure network nic create \
	--resource-group the-hard-way \
	--name etcd-0-nic \
	--private-ip-address "10.240.0.10" \
	--subnet-vnet-name the-hard-way-net  \
	--subnet-name kubernetes  \
	--location "West Us"
```
Create VM

```
azure vm create \
	--resource-group the-hard-way \
	--name etcd0 \
	--vm-size Standard_A1 \
	--nic-name etcd-0-nic \
	--vnet-name the-hard-way-net  \
	--vnet-subnet-name kubernetes  \
	--os-type linux \
	--image-urn $imageUrn \
	--storage-account-name $controlPlaneStorageAccount \
	--storage-account-container-name vhds \
	--os-disk-vhd etcd-0.vhd \
	--admin-username thehardway \
	--ssh-publickey-file ./keys/cluster.pub \
	--location "West US"
```

#### Etcd 1

Create Nic

```
azure network nic create \
	--resource-group the-hard-way \
	--name etcd-1-nic \
	--private-ip-address "10.240.0.11" \
	--subnet-vnet-name the-hard-way-net  \
	--subnet-name kubernetes  \
	--location "West Us"
```

Create VM

```
azure vm create \
	--resource-group the-hard-way \
	--name etcd1 \
	--vm-size Standard_A1 \
	--nic-name etcd-1-nic \
	--vnet-name the-hard-way-net  \
	--vnet-subnet-name kubernetes  \
	--os-type linux \
	--image-urn $imageUrn \
	--storage-account-name $controlPlaneStorageAccount \
	--storage-account-container-name vhds \
	--os-disk-vhd etcd-1.vhd \
	--admin-username thehardway \
	--ssh-publickey-file ./keys/cluster.pub \
	--location "West US"
```

#### Etcd 2

Create Nic

```
azure network nic create \
	--resource-group the-hard-way \
	--name etcd-2-nic \
	--private-ip-address "10.240.0.12" \
	--subnet-vnet-name the-hard-way-net  \
	--subnet-name kubernetes  \
	--location "West Us"
```

Create VM

```
azure vm create \
	--resource-group the-hard-way \
	--name etcd2 \
	--vm-size Standard_A1 \
	--nic-name etcd-2-nic \
	--vnet-name the-hard-way-net  \
	--vnet-subnet-name kubernetes  \
	--os-type linux \
	--image-urn $imageUrn \
	--storage-account-name $controlPlaneStorageAccount \
	--storage-account-container-name vhds \
	--os-disk-vhd etcd-2.vhd \
	--admin-username thehardway \
	--ssh-publickey-file ./keys/cluster.pub \
	--location "West US"
```


### Kubernetes Controllers


#### Controllers Internal Load Balancer 

Create load balancer 

```
azure network lb create \
	--resource-group the-hard-way \
	--name the-hard-way-clb \
	--location "West Us"
```

Create & assign the front-end private IP to the internal load balancer

```
azure network lb frontend-ip create \
	--resource-group the-hard-way \
	--name the-hard-way-cfe  \
	--lb-name the-hard-way-clb \
	--private-ip-address "10.240.0.4" \
	--subnet-vnet-name the-hard-way-net \
	--subnet-name kubernetes
```

Create a backend address pool for the load balancer

```
clbbackendPoolId=$(azure network lb address-pool create \
	--resource-group the-hard-way \
	--lb-name the-hard-way-clb \
	--name backend-pool \
	--json | jq -r '.id')
```

#### Create controllers availability set

```
azure availset create \
	--resource-group the-hard-way \
	--name controllers-availset \
	--location "West Us"
```


#### Controller 0 

Create Nic

```
azure network nic create \
	--resource-group the-hard-way \
	--name controller-0-nic \
	--private-ip-address "10.240.0.20" \
	--subnet-vnet-name the-hard-way-net  \
	--subnet-name kubernetes  \
	--lb-address-pool-ids $clbbackendPoolId \
	--location "West Us"
```

Create VM

```
azure vm create \
	--resource-group the-hard-way \
    --name controller0 \
    --vm-size Standard_A1 \
    --nic-name controller-0-nic \
	--vnet-name the-hard-way-net  \
	--vnet-subnet-name kubernetes  \
	--availset-name controllers-availset \
    --os-type linux \
    --image-urn $imageUrn \
    --storage-account-name $controlPlaneStorageAccount \
    --storage-account-container-name vhds \
    --os-disk-vhd controller-0.vhd \
    --admin-username thehardway \
    --ssh-publickey-file ./keys/cluster.pub \
	--location "West US"
```

#### Controller 1 

Create Nic

```
azure network nic create \
	--resource-group the-hard-way \
	--name controller-1-nic \
	--private-ip-address "10.240.0.21" \
	--subnet-vnet-name the-hard-way-net  \
	--subnet-name kubernetes  \
	--lb-address-pool-ids $clbbackendPoolId \
	--location "West Us"
```

Create VM

```
azure vm create \
	--resource-group the-hard-way \
    --name controller1 \
    --vm-size Standard_A1 \
    --nic-name controller-1-nic \
	--vnet-name the-hard-way-net  \
	--vnet-subnet-name kubernetes  \
	--availset-name controllers-availset \
    --os-type linux \
    --image-urn $imageUrn \
    --storage-account-name $controlPlaneStorageAccount \
    --storage-account-container-name vhds \
    --os-disk-vhd controller-1.vhd \
    --admin-username thehardway \
    --ssh-publickey-file ./keys/cluster.pub \
	--location "West US"
```

#### Controller 2

Create Nic

```
azure network nic create \
	--resource-group the-hard-way \
	--name controller-2-nic \
	--private-ip-address "10.240.0.22" \
	--subnet-vnet-name the-hard-way-net  \
	--subnet-name kubernetes  \
	--lb-address-pool-ids $clbbackendPoolId \
	--location "West Us"
```

Create VM

```
azure vm create \
	--resource-group the-hard-way \
    --name controller2 \
    --vm-size Standard_A1 \
    --nic-names controller-2-nic \
	--vnet-name the-hard-way-net  \
	--vnet-subnet-name kubernetes  \
	--availset-name controllers-availset \
    --os-type linux \
    --image-urn $imageUrn \
    --storage-account-name $controlPlaneStorageAccount \
    --storage-account-container-name vhds \
    --os-disk-vhd controller-2.vhd \
    --admin-username thehardway \
    --ssh-publickey-file ./keys/cluster.pub \
	--location "West US"
```


### Kubernetes Workers

#### Workers External Load Balancer 

Create public IP + DNS label for workers ingestion external load balancer

```
azure network public-ip create \
	--resource-group the-hard-way \
	--name the-hard-way-workers  \
	--allocation-method Dynamic \
	--domain-name-label $workersDnsLabel \
	--location "West Us"
```

Create load balancer 

```
azure network lb create \
	--resource-group the-hard-way \
	--name the-hard-way-lb \
	--location "West Us"
```

Assign the front-end public IP + FQDN to the load balancer

```
azure network lb frontend-ip create \
	--resource-group the-hard-way \
	--name the-hard-way-fe  \
	--lb-name the-hard-way-lb \
	--public-ip-name the-hard-way-workers
```

Create a backend address pool for the load balancer

```
wlbbackendPoolId=$(azure network lb address-pool create \
	--resource-group the-hard-way \
	--lb-name the-hard-way-lb \
	--name backend-pool \
	--json | jq -r '.id')
```

#### Create Workers Availablity set

```
azure availset create \
	--resource-group the-hard-way \
	--name workers-availset \
	--location "West Us"
```

#### Worker 0

Create Nic

```
azure network nic create \
	--resource-group the-hard-way \
	--name worker-0-nic \
	--private-ip-address "10.240.0.30" \
	--subnet-vnet-name the-hard-way-net  \
	--subnet-name kubernetes  \
	--enable-ip-forwarding "true" \
	--lb-address-pool-ids $wlbbackendPoolId \
	--location "West Us"
```

Create VM

```
azure vm create \
	--resource-group the-hard-way \
    --name worker0 \
    --vm-size Standard_D4 \
    --nic-name worker-0-nic \
	--vnet-name the-hard-way-net  \
	--vnet-subnet-name kubernetes  \
    --availset-name workers-availset \
	--os-type linux \
    --image-urn $imageUrn \
    --storage-account-name $workersStorageAccount \
    --storage-account-container-name vhds \
    --os-disk-vhd worker-0.vhd \
    --admin-username thehardway \
    --ssh-publickey-file ./keys/cluster.pub \
	--location "West US"
```

#### Worker 1

Create Nic

```
azure network nic create \
	--resource-group the-hard-way \
	--name worker-1-nic \
	--private-ip-address "10.240.0.31" \
	--subnet-vnet-name the-hard-way-net  \
	--subnet-name kubernetes  \
	--enable-ip-forwarding "true" \
	--lb-address-pool-ids $wlbbackendPoolId \
	--location "West Us"
```

Create VM

```
azure vm create \
	--resource-group the-hard-way \
    --name worker1 \
    --vm-size Standard_D4 \
    --nic-name worker-1-nic \
	--vnet-name the-hard-way-net  \
	--vnet-subnet-name kubernetes  \
    --availset-name workers-availset \
	--os-type linux \
    --image-urn $imageUrn \
    --storage-account-name $workersStorageAccount \
    --storage-account-container-name vhds \
    --os-disk-vhd worker-1.vhd \
    --admin-username thehardway \
    --ssh-publickey-file ./keys/cluster.pub \
	--location "West US"
```

#### Worker 2

Create Nic

```
azure network nic create \
	--resource-group the-hard-way \
	--name worker-2-nic \
	--private-ip-address "10.240.0.32" \
	--subnet-vnet-name the-hard-way-net  \
	--subnet-name kubernetes  \
	--enable-ip-forwarding "true" \
	--lb-address-pool-ids $wlbbackendPoolId \
	--location "West Us"
```

Create VM

```
azure vm create \
	--resource-group the-hard-way \
    --name worker2 \
    --vm-size Standard_D4 \
    --nic-name worker-2-nic \
	--vnet-name the-hard-way-net  \
	--vnet-subnet-name kubernetes  \
    --availset-name workers-availset \
	--os-type linux \
    --image-urn $imageUrn \
    --storage-account-name $workersStorageAccount \
    --storage-account-container-name vhds \
    --os-disk-vhd worker-2.vhd \
    --admin-username thehardway \
    --ssh-publickey-file ./keys/cluster.pub \
	--location "West US"
```

## Verify 

```
azure vm list --resource-group the-hard-way
```

Expected Output 
```
info:    Executing command vm list
+ Getting virtual machines                                                     
data:    ResourceGroupName  Name          ProvisioningState  PowerState  Location  Size       
data:    -----------------  ------------  -----------------  ----------  --------  -----------
data:    the-hard-way       controller-0  Succeeded          VM running  westus    Standard_A1
data:    the-hard-way       controller-1  Succeeded          VM running  westus    Standard_A1
data:    the-hard-way       controller-2  Succeeded          VM running  westus    Standard_A1
data:    the-hard-way       etcd-0        Succeeded          VM running  westus    Standard_A1
data:    the-hard-way       etcd-1        Succeeded          VM running  westus    Standard_A1
data:    the-hard-way       etcd-2        Succeeded          VM running  westus    Standard_A1
data:    the-hard-way       jumpbox       Succeeded          VM running  westus    Standard_A1
data:    the-hard-way       worker-0      Succeeded          VM running  westus    Standard_A1
data:    the-hard-way       worker-1      Succeeded          VM running  westus    Standard_A1
data:    the-hard-way       worker-2      Succeeded          VM running  westus    Standard_A1
info:    vm list command OK
```


## Using The Jumpbox

> The Jumpbox does not have Azure CLI installed. All further Azure CLI commands should be executed on a machine with CLI installed & configured.   

### Connect to Jumpbox 

```
ssh -i ./keys/cluster \
	thehardway@$jumpboxDnsLabel.westus.cloudapp.azure.com
```

### Copy the cluster private key to jumpbox 

```
scp -i ./keys/cluster \
	./keys/cluster \
	thehardway@$jumpboxDnsLabel.westus.cloudapp.azure.com:~/cluster
```

### Connecting to Other VMs

```
#from the jumpbox 
#connect to the second controller

ssh -i ./cluster \
	thehardway@10.240.0.31  

#or

ssh -i ./cluster \
	thehardway@controller-1  

```