diff --git a/README.md b/README.md index bb8d14c..c1edea1 100644 --- a/README.md +++ b/README.md @@ -6,6 +6,7 @@ This tutorial is optimized for learning, which means taking the long route to he * [Google Compute Engine](https://cloud.google.com/compute) * [Amazon EC2](https://aws.amazon.com/ec2) +* [Microsoft Azure VMs](https://azure.microsoft.com/en-us/services/virtual-machines/?b=16.51b) > The results of this tutorial should not be viewed as production ready, and may receive limited support from the community, but don't let that prevent you from learning! @@ -48,16 +49,21 @@ AWS * The us-west-2 region will be used * ``jq`` parsing requires [AWS CLI output format](http://docs.aws.amazon.com/cli/latest/userguide/controlling-output.html) to be ``json`` +Azure + +* The westus region will be used + ## Platforms This tutorial assumes you have access to one of the following: * [Google Cloud Platform](https://cloud.google.com) and the [Google Cloud SDK](https://cloud.google.com/sdk/) (125.0.0+) * [Amazon Web Services](https://aws.amazon.com), the [AWS CLI](https://aws.amazon.com/cli) (1.10.63+), and [jq](https://stedolan.github.io/jq) (1.5+) +* [Microsoft Azure](https://azure.com) and the [Azure CLI 2.0](https://github.com/azure/azure-cli) ## Labs -While GCP or AWS will be used for basic infrastructure needs, the things learned in this tutorial apply to every platform. +While GCP, AWS or Azure will be used for basic infrastructure needs, the things learned in this tutorial apply to every platform. * [Cloud Infrastructure Provisioning](docs/01-infrastructure.md) * [Setting up a CA and TLS Cert Generation](docs/02-certificate-authority.md) diff --git a/docs/01-infrastructure-azure.md b/docs/01-infrastructure-azure.md new file mode 100644 index 0000000..5d7159d --- /dev/null +++ b/docs/01-infrastructure-azure.md @@ -0,0 +1,175 @@ +# Cloud Infrastructure Provisioning - Microsoft Azure + +This lab will walk you through provisioning the compute instances required for running a H/A Kubernetes cluster. A total of 6 virtual machines will be created. + +The guide assumes you've installed the [Azure CLI 2.0](https://github.com/azure/azure-cli#installation), and will be creating resources in the `westus` region, within a resource group named `kubernetes`. To create this resource group, simply run the following command: + +```shell +az group create -n kubernetes -l westus +``` + +After completing this guide you should have the following compute instances: + +```shell +az vm list --query "[].{name:name,provisioningState:provisioningState}" +``` + +```shell +Name ProvisioningState +----------- ------------------- +controller0 Succeeded +controller1 Succeeded +controller2 Succeeded +worker0 Succeeded +worker1 Succeeded +worker2 Succeeded +``` + +> All machines will be provisioned with fixed private IP addresses to simplify the bootstrap process. + +To make our Kubernetes control plane remotely accessible, a public IP address will be provisioned and assigned to a load balancer that will sit in front of the 3 Kubernetes controllers. + +## Networking + +Create a virtual network and subnet for the Kubernetes cluster: + +```shell +az network vnet create -g kubernetes \ + -n kubernetes-vnet \ + --address-prefix 10.240.0.0/16 \ + --subnet-name kubernetes-subnet +``` + +### Firewall Rules + +Create a firewall ("network security group"), assign it to the subnet, and configure it to allow the necessary incoming traffic: + +```shell +az network nsg create -g kubernetes -n kubernetes-nsg +``` + +```shell +az network vnet subnet update -g kubernetes \ + -n kubernetes-subnet \ + --vnet-name kubernetes-vnet \ + --network-security-group kubernetes-nsg +``` + +```shell +az network nsg rule create -g kubernetes \ + -n kubernetes-allow-ssh \ + --access allow \ + --destination-address-prefix '*' \ + --destination-port-range 22 \ + --direction inbound \ + --nsg-name kubernetes-nsg \ + --protocol tcp \ + --source-address-prefix '*' \ + --source-port-range '*' \ + --priority 1000 +``` + +```shell +az network nsg rule create -g kubernetes \ + -n kubernetes-allow-api-server \ + --access allow \ + --destination-address-prefix '*' \ + --destination-port-range 6443 \ + --direction inbound \ + --nsg-name kubernetes-nsg \ + --protocol tcp \ + --source-address-prefix '*' \ + --source-port-range '*' \ + --priority 1001 +``` + +```shell +az network nsg rule list -g kubernetes --nsg-name kubernetes-nsg --query "[].{Name:name, Port:destinationPortRange}" +``` + +``` +Name Port +--------------------- ------ +kube-allow-ssh 22 +kube-allow-api-server 6443 +``` + +### Kubernetes Public Address + +Create a public IP address that will be used by remote clients to connect to the Kubernetes control plane: + +```shell +az network lb create -g kubernetes \ + -n kubernetes-lb \ + --backend-pool-name kubernetes-lb-pool \ + --public-ip-address kubernetes-pip \ + --public-ip-address-allocation static +``` + +## Provision Virtual Machines + +All the VMs in this lab will be provisioned using Ubuntu 16.04 mainly because it runs a newish Linux Kernel that has good support for Docker. + +### Virtual Machines + +#### Kubernetes Controllers + +```shell +az vm availability-set create -g kubernetes -n controller-as +``` + +```shell +for num in {0..2}; do + echo "[Controller ${num}] Creating public IP..." + az network public-ip create -n controller${num}-pip -g kubernetes > /dev/null + + echo "[Controller ${num}] Creating NIC..." + az network nic create -g kubernetes \ + -n controller${num}-nic \ + --private-ip-address 10.240.0.1${num} \ + --public-ip-address controller${num}-pip \ + --vnet kubernetes-vnet \ + --subnet kubernetes-subnet \ + --ip-forwarding \ + --lb-name kubernetes-lb \ + --lb-address-pools kubernetes-lb-pool > /dev/null + + echo "[Controller ${num}] Creating VM..." + az vm create -g kubernetes \ + -n controller${num} \ + --image Canonical:UbuntuServer:16.04.0-LTS:16.04.201609210 \ + --nics controller${num}-nic \ + --availability-set controller-as \ + --nsg '' > /dev/null +done +``` + +#### Kubernetes Workers + +```shell +az vm availability-set create -g kubernetes -n worker-as +``` + +```shell +for num in {0..2}; do + echo "[Worker ${num}] Creating public IP..." + az network public-ip create -n worker${num}-pip -g kubernetes > /dev/null + + echo "[Worker ${num}] Creating NIC..." + az network nic create -g kubernetes \ + -n worker${num}-nic \ + --private-ip-address 10.240.0.2${num} \ + --public-ip-address worker${num}-pip \ + --vnet kubernetes-vnet \ + --subnet kubernetes-subnet \ + --ip-forwarding > /dev/null + + echo "[Worker ${num}] Creating VM..." + az vm create -g kubernetes \ + -n worker${num} \ + --image Canonical:UbuntuServer:16.04.0-LTS:16.04.201609210 \ + --nics worker${num}-nic \ + --availability-set worker-as \ + --nsg '' > /dev/null +done +``` \ No newline at end of file diff --git a/docs/01-infrastructure.md b/docs/01-infrastructure.md index bd4ae69..6c5c9c1 100644 --- a/docs/01-infrastructure.md +++ b/docs/01-infrastructure.md @@ -1,8 +1,9 @@ # Cloud Infrastructure Provisioning -Kubernetes can be installed just about anywhere physical or virtual machines can be run. In this lab we are going to focus on [Google Cloud Platform](https://cloud.google.com/) and [Amazon Web Services](https://aws.amazon.com). +Kubernetes can be installed just about anywhere physical or virtual machines can be run. In this lab we are going to focus on [Google Cloud Platform](https://cloud.google.com/), [Amazon Web Services](https://aws.amazon.com) and [Microsoft Azure](https://azure.com). This lab will walk you through provisioning the compute instances required for running a H/A Kubernetes cluster. * [Cloud Infrastructure Provisioning - Google Cloud Platform](01-infrastructure-gcp.md) * [Cloud Infrastructure Provisioning - Amazon Web Services](01-infrastructure-aws.md) +* [Cloud Infrastructure Provisioning - Microsoft Azure](01-infrastructure-azure.md) \ No newline at end of file diff --git a/docs/02-certificate-authority.md b/docs/02-certificate-authority.md index 80f7572..e8e81d8 100644 --- a/docs/02-certificate-authority.md +++ b/docs/02-certificate-authority.md @@ -38,7 +38,6 @@ chmod +x cfssljson_darwin-amd64 sudo mv cfssljson_darwin-amd64 /usr/local/bin/cfssljson ``` - ### Linux ``` @@ -137,6 +136,13 @@ KUBERNETES_PUBLIC_ADDRESS=$(aws elb describe-load-balancers \ jq -r '.LoadBalancerDescriptions[].DNSName') ``` +#### Azure + +```shell +KUBERNETES_PUBLIC_ADDRESS=$(az network public-ip show -g kubernetes \ + -n kubernetes-pip --query "ipAddress" -otsv) +``` + --- Create the `kubernetes-csr.json` file: @@ -230,7 +236,7 @@ done The following command will: * Extract the public IP address for each Kubernetes host * Copy the TLS certificates and keys to each Kubernetes host using `scp` - + ``` for host in ${KUBERNETES_HOSTS[*]}; do PUBLIC_IP_ADDRESS=$(aws ec2 describe-instances \ @@ -240,3 +246,19 @@ for host in ${KUBERNETES_HOSTS[*]}; do ubuntu@${PUBLIC_IP_ADDRESS}:~/ done ``` + +### Azure + +The following command will: + * Extract the public IP address for each Kubernetes host + * Copy the TLS certificates and keys to each Kubernetes host using `scp` + +```shell +for host in ${KUBERNETES_HOSTS[*]}; do + PUBLIC_IP_ADDRESS=$(az network public-ip show -g kubernetes \ + -n ${host}-pip --query "ipAddress" -otsv) + + scp ca.pem kubernetes-key.pem kubernetes.pem \ + $(whoami)@${PUBLIC_IP_ADDRESS}:~/ +done +``` \ No newline at end of file diff --git a/docs/03-etcd.md b/docs/03-etcd.md index 1321992..a07df8a 100644 --- a/docs/03-etcd.md +++ b/docs/03-etcd.md @@ -108,6 +108,12 @@ INTERNAL_IP=$(curl -s -H "Metadata-Flavor: Google" \ INTERNAL_IP=$(curl -s http://169.254.169.254/latest/meta-data/local-ipv4) ``` +#### Azure + +```shell +INTERNAL_IP=$(ip addr show eth0 | grep -oP '(?<=inet\s)\d+(\.\d+){3}') +``` + --- Each etcd member must have a unique name within an etcd cluster. Set the etcd name: @@ -168,4 +174,4 @@ member 3a57933972cb5131 is healthy: got healthy result from https://10.240.0.12: member f98dc20bce6225a0 is healthy: got healthy result from https://10.240.0.10:2379 member ffed16798470cab5 is healthy: got healthy result from https://10.240.0.11:2379 cluster is healthy -``` +``` \ No newline at end of file diff --git a/docs/04-kubernetes-controller.md b/docs/04-kubernetes-controller.md index 2a33b25..aff59ff 100644 --- a/docs/04-kubernetes-controller.md +++ b/docs/04-kubernetes-controller.md @@ -138,6 +138,12 @@ INTERNAL_IP=$(curl -s -H "Metadata-Flavor: Google" \ INTERNAL_IP=$(curl -s http://169.254.169.254/latest/meta-data/local-ipv4) ``` +#### Azure + +```shell +INTERNAL_IP=$(ip addr show eth0 | grep -oP '(?<=inet\s)\d+(\.\d+){3}') +``` + --- Create the systemd unit file: @@ -234,7 +240,6 @@ sed -i s/INTERNAL_IP/$INTERNAL_IP/g kube-controller-manager.service sudo mv kube-controller-manager.service /etc/systemd/system/ ``` - ``` sudo systemctl daemon-reload sudo systemctl enable kube-controller-manager @@ -343,3 +348,26 @@ aws elb register-instances-with-load-balancer \ --load-balancer-name kubernetes \ --instances ${CONTROLLER_0_INSTANCE_ID} ${CONTROLLER_1_INSTANCE_ID} ${CONTROLLER_2_INSTANCE_ID} ``` + +### Azure + +```shell +az network lb probe create -g kubernetes \ + -n kubernetes-apiserver-check \ + --lb-name kubernetes-lb \ + --protocol http \ + --port 8080 \ + --path /healthz +``` + +```shell +az network lb rule create -g kubernetes \ + -n kubernetes-apiserver-rule \ + --protocol tcp \ + --lb-name kubernetes-lb \ + --frontend-ip-name LoadBalancerFrontEnd \ + --frontend-port 6443 \ + --backend-pool-name kubernetes-lb-pool \ + --backend-port 6443 \ + --probe-name kubernetes-apiserver-check +``` \ No newline at end of file diff --git a/docs/05-kubernetes-worker.md b/docs/05-kubernetes-worker.md index 08ee9f1..35545b0 100644 --- a/docs/05-kubernetes-worker.md +++ b/docs/05-kubernetes-worker.md @@ -15,7 +15,6 @@ Kubernetes worker nodes are responsible for running your containers. All Kuberne Some people would like to run workers and cluster services anywhere in the cluster. This is totally possible, and you'll have to decide what's best for your environment. - ## Provision the Kubernetes Worker Nodes Run the following commands on `worker0`, `worker1`, `worker2`: diff --git a/docs/06-kubectl.md b/docs/06-kubectl.md index 65bbe1f..15383c2 100644 --- a/docs/06-kubectl.md +++ b/docs/06-kubectl.md @@ -38,6 +38,14 @@ KUBERNETES_PUBLIC_ADDRESS=$(aws elb describe-load-balancers \ --load-balancer-name kubernetes | \ jq -r '.LoadBalancerDescriptions[].DNSName') ``` + +#### Azure + +```shell +KUBERNETES_PUBLIC_ADDRESS=$(az network public-ip show -g kubernetes \ + -n kubernetes-pip --query "ipAddress" -otsv) +``` + --- Recall the token we setup for the admin user: @@ -97,4 +105,4 @@ NAME STATUS AGE worker0 Ready 7m worker1 Ready 5m worker2 Ready 2m -``` +``` \ No newline at end of file diff --git a/docs/07-network.md b/docs/07-network.md index 5b99809..31e6657 100644 --- a/docs/07-network.md +++ b/docs/07-network.md @@ -118,3 +118,44 @@ aws ec2 create-route \ --destination-cidr-block 10.200.2.0/24 \ --instance-id ${WORKER_2_INSTANCE_ID} ``` + +### Azure + +```shell +az network route-table create -g kubernetes \ + -n kubernetes-routes +``` + +```shell +az network vnet subnet update -g kubernetes \ + -n kubernetes-subnet \ + --vnet-name kubernetes-vnet \ + --route-table kubernetes-routes +``` + +```shell +az network route-table route create -g kubernetes \ + -n kubernetes-route-10-200-0-0-24 \ + --route-table-name kubernetes-routes \ + --address-prefix 10.200.0.0/24 \ + --next-hop-ip-address 10.240.0.20 \ + --next-hop-type VirtualAppliance +``` + +```shell +az network route-table route create -g kubernetes \ + -n kubernetes-route-10-200-1-0-24 \ + --route-table-name kubernetes-routes \ + --address-prefix 10.200.1.0/24 \ + --next-hop-ip-address 10.240.0.21 \ + --next-hop-type VirtualAppliance +``` + +```shell +az network route-table route create -g kubernetes \ + -n kubernetes-route-10-200-2-0-24 \ + --route-table-name kubernetes-routes \ + --address-prefix 10.200.2.0/24 \ + --next-hop-ip-address 10.240.0.22 \ + --next-hop-type VirtualAppliance +``` \ No newline at end of file diff --git a/docs/09-smoke-test.md b/docs/09-smoke-test.md index a812108..f361624 100644 --- a/docs/09-smoke-test.md +++ b/docs/09-smoke-test.md @@ -79,6 +79,29 @@ NODE_PUBLIC_IP=$(aws ec2 describe-instances \ jq -j '.Reservations[].Instances[].PublicIpAddress') ``` +#### Azure + +```shell +az network nsg rule create -g kubernetes \ + -n kubernetes-allow-nginx \ + --access allow \ + --destination-address-prefix '*' \ + --destination-port-range ${NODE_PORT} \ + --direction inbound \ + --nsg-name kubernetes-nsg \ + --protocol tcp \ + --source-address-prefix '*' \ + --source-port-range '*' \ + --priority 1002 +``` + +Grab the `EXTERNAL_IP` for one of the worker nodes: + +``` +NODE_PUBLIC_IP=$(az network public-ip show -g kubernetes \ + -n worker0-pip --query "ipAddress" -otsv) +``` + --- Test the nginx service using cURL: diff --git a/docs/10-cleanup.md b/docs/10-cleanup.md index 452b58a..8289480 100644 --- a/docs/10-cleanup.md +++ b/docs/10-cleanup.md @@ -205,3 +205,9 @@ DHCP_OPTION_SET_ID=$(aws ec2 describe-dhcp-options \ aws ec2 delete-dhcp-options \ --dhcp-options-id ${DHCP_OPTION_SET_ID} ``` + +## Azure + +```shell +az group delete -n kubernetes +```