Merge 9f31aca0e4 into b7ae204c86
Jonathan Carter
GitHub
commit
ae3e8c29c1
|
|
@ -6,6 +6,7 @@ This tutorial is optimized for learning, which means taking the long route to he
|
|||
|
||||
* [Google Compute Engine](https://cloud.google.com/compute)
|
||||
* [Amazon EC2](https://aws.amazon.com/ec2)
|
||||
* [Microsoft Azure VMs](https://azure.microsoft.com/en-us/services/virtual-machines/?b=16.51b)
|
||||
|
||||
> The results of this tutorial should not be viewed as production ready, and may receive limited support from the community, but don't let that prevent you from learning!
|
||||
|
||||
|
|
@ -48,16 +49,21 @@ AWS
|
|||
* The us-west-2 region will be used
|
||||
* ``jq`` parsing requires [AWS CLI output format](http://docs.aws.amazon.com/cli/latest/userguide/controlling-output.html) to be ``json``
|
||||
|
||||
Azure
|
||||
|
||||
* The westus region will be used
|
||||
|
||||
## Platforms
|
||||
|
||||
This tutorial assumes you have access to one of the following:
|
||||
|
||||
* [Google Cloud Platform](https://cloud.google.com) and the [Google Cloud SDK](https://cloud.google.com/sdk/) (125.0.0+)
|
||||
* [Amazon Web Services](https://aws.amazon.com), the [AWS CLI](https://aws.amazon.com/cli) (1.10.63+), and [jq](https://stedolan.github.io/jq) (1.5+)
|
||||
* [Microsoft Azure](https://azure.com) and the [Azure CLI 2.0](https://github.com/azure/azure-cli)
|
||||
|
||||
## Labs
|
||||
|
||||
While GCP or AWS will be used for basic infrastructure needs, the things learned in this tutorial apply to every platform.
|
||||
While GCP, AWS or Azure will be used for basic infrastructure needs, the things learned in this tutorial apply to every platform.
|
||||
|
||||
* [Cloud Infrastructure Provisioning](docs/01-infrastructure.md)
|
||||
* [Setting up a CA and TLS Cert Generation](docs/02-certificate-authority.md)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,175 @@
|
|||
# Cloud Infrastructure Provisioning - Microsoft Azure
|
||||
|
||||
This lab will walk you through provisioning the compute instances required for running a H/A Kubernetes cluster. A total of 6 virtual machines will be created.
|
||||
|
||||
The guide assumes you've installed the [Azure CLI 2.0](https://github.com/azure/azure-cli#installation), and will be creating resources in the `westus` region, within a resource group named `kubernetes`. To create this resource group, simply run the following command:
|
||||
|
||||
```shell
|
||||
az group create -n kubernetes -l westus
|
||||
```
|
||||
|
||||
After completing this guide you should have the following compute instances:
|
||||
|
||||
```shell
|
||||
az vm list --query "[].{name:name,provisioningState:provisioningState}"
|
||||
```
|
||||
|
||||
```shell
|
||||
Name ProvisioningState
|
||||
----------- -------------------
|
||||
controller0 Succeeded
|
||||
controller1 Succeeded
|
||||
controller2 Succeeded
|
||||
worker0 Succeeded
|
||||
worker1 Succeeded
|
||||
worker2 Succeeded
|
||||
```
|
||||
|
||||
> All machines will be provisioned with fixed private IP addresses to simplify the bootstrap process.
|
||||
|
||||
To make our Kubernetes control plane remotely accessible, a public IP address will be provisioned and assigned to a load balancer that will sit in front of the 3 Kubernetes controllers.
|
||||
|
||||
## Networking
|
||||
|
||||
Create a virtual network and subnet for the Kubernetes cluster:
|
||||
|
||||
```shell
|
||||
az network vnet create -g kubernetes \
|
||||
-n kubernetes-vnet \
|
||||
--address-prefix 10.240.0.0/16 \
|
||||
--subnet-name kubernetes-subnet
|
||||
```
|
||||
|
||||
### Firewall Rules
|
||||
|
||||
Create a firewall ("network security group"), assign it to the subnet, and configure it to allow the necessary incoming traffic:
|
||||
|
||||
```shell
|
||||
az network nsg create -g kubernetes -n kubernetes-nsg
|
||||
```
|
||||
|
||||
```shell
|
||||
az network vnet subnet update -g kubernetes \
|
||||
-n kubernetes-subnet \
|
||||
--vnet-name kubernetes-vnet \
|
||||
--network-security-group kubernetes-nsg
|
||||
```
|
||||
|
||||
```shell
|
||||
az network nsg rule create -g kubernetes \
|
||||
-n kubernetes-allow-ssh \
|
||||
--access allow \
|
||||
--destination-address-prefix '*' \
|
||||
--destination-port-range 22 \
|
||||
--direction inbound \
|
||||
--nsg-name kubernetes-nsg \
|
||||
--protocol tcp \
|
||||
--source-address-prefix '*' \
|
||||
--source-port-range '*' \
|
||||
--priority 1000
|
||||
```
|
||||
|
||||
```shell
|
||||
az network nsg rule create -g kubernetes \
|
||||
-n kubernetes-allow-api-server \
|
||||
--access allow \
|
||||
--destination-address-prefix '*' \
|
||||
--destination-port-range 6443 \
|
||||
--direction inbound \
|
||||
--nsg-name kubernetes-nsg \
|
||||
--protocol tcp \
|
||||
--source-address-prefix '*' \
|
||||
--source-port-range '*' \
|
||||
--priority 1001
|
||||
```
|
||||
|
||||
```shell
|
||||
az network nsg rule list -g kubernetes --nsg-name kubernetes-nsg --query "[].{Name:name, Port:destinationPortRange}"
|
||||
```
|
||||
|
||||
```
|
||||
Name Port
|
||||
--------------------- ------
|
||||
kube-allow-ssh 22
|
||||
kube-allow-api-server 6443
|
||||
```
|
||||
|
||||
### Kubernetes Public Address
|
||||
|
||||
Create a public IP address that will be used by remote clients to connect to the Kubernetes control plane:
|
||||
|
||||
```shell
|
||||
az network lb create -g kubernetes \
|
||||
-n kubernetes-lb \
|
||||
--backend-pool-name kubernetes-lb-pool \
|
||||
--public-ip-address kubernetes-pip \
|
||||
--public-ip-address-allocation static
|
||||
```
|
||||
|
||||
## Provision Virtual Machines
|
||||
|
||||
All the VMs in this lab will be provisioned using Ubuntu 16.04 mainly because it runs a newish Linux Kernel that has good support for Docker.
|
||||
|
||||
### Virtual Machines
|
||||
|
||||
#### Kubernetes Controllers
|
||||
|
||||
```shell
|
||||
az vm availability-set create -g kubernetes -n controller-as
|
||||
```
|
||||
|
||||
```shell
|
||||
for num in {0..2}; do
|
||||
echo "[Controller ${num}] Creating public IP..."
|
||||
az network public-ip create -n controller${num}-pip -g kubernetes > /dev/null
|
||||
|
||||
echo "[Controller ${num}] Creating NIC..."
|
||||
az network nic create -g kubernetes \
|
||||
-n controller${num}-nic \
|
||||
--private-ip-address 10.240.0.1${num} \
|
||||
--public-ip-address controller${num}-pip \
|
||||
--vnet kubernetes-vnet \
|
||||
--subnet kubernetes-subnet \
|
||||
--ip-forwarding \
|
||||
--lb-name kubernetes-lb \
|
||||
--lb-address-pools kubernetes-lb-pool > /dev/null
|
||||
|
||||
echo "[Controller ${num}] Creating VM..."
|
||||
az vm create -g kubernetes \
|
||||
-n controller${num} \
|
||||
--image Canonical:UbuntuServer:16.04.0-LTS:16.04.201609210 \
|
||||
--nics controller${num}-nic \
|
||||
--availability-set controller-as \
|
||||
--nsg '' > /dev/null
|
||||
done
|
||||
```
|
||||
|
||||
#### Kubernetes Workers
|
||||
|
||||
```shell
|
||||
az vm availability-set create -g kubernetes -n worker-as
|
||||
```
|
||||
|
||||
```shell
|
||||
for num in {0..2}; do
|
||||
echo "[Worker ${num}] Creating public IP..."
|
||||
az network public-ip create -n worker${num}-pip -g kubernetes > /dev/null
|
||||
|
||||
echo "[Worker ${num}] Creating NIC..."
|
||||
az network nic create -g kubernetes \
|
||||
-n worker${num}-nic \
|
||||
--private-ip-address 10.240.0.2${num} \
|
||||
--public-ip-address worker${num}-pip \
|
||||
--vnet kubernetes-vnet \
|
||||
--subnet kubernetes-subnet \
|
||||
--ip-forwarding > /dev/null
|
||||
|
||||
echo "[Worker ${num}] Creating VM..."
|
||||
az vm create -g kubernetes \
|
||||
-n worker${num} \
|
||||
--image Canonical:UbuntuServer:16.04.0-LTS:16.04.201609210 \
|
||||
--nics worker${num}-nic \
|
||||
--availability-set worker-as \
|
||||
--nsg '' > /dev/null
|
||||
done
|
||||
```
|
||||
|
|
@ -1,8 +1,9 @@
|
|||
# Cloud Infrastructure Provisioning
|
||||
|
||||
Kubernetes can be installed just about anywhere physical or virtual machines can be run. In this lab we are going to focus on [Google Cloud Platform](https://cloud.google.com/) and [Amazon Web Services](https://aws.amazon.com).
|
||||
Kubernetes can be installed just about anywhere physical or virtual machines can be run. In this lab we are going to focus on [Google Cloud Platform](https://cloud.google.com/), [Amazon Web Services](https://aws.amazon.com) and [Microsoft Azure](https://azure.com).
|
||||
|
||||
This lab will walk you through provisioning the compute instances required for running a H/A Kubernetes cluster.
|
||||
|
||||
* [Cloud Infrastructure Provisioning - Google Cloud Platform](01-infrastructure-gcp.md)
|
||||
* [Cloud Infrastructure Provisioning - Amazon Web Services](01-infrastructure-aws.md)
|
||||
* [Cloud Infrastructure Provisioning - Microsoft Azure](01-infrastructure-azure.md)
|
||||
|
|
@ -38,7 +38,6 @@ chmod +x cfssljson_darwin-amd64
|
|||
sudo mv cfssljson_darwin-amd64 /usr/local/bin/cfssljson
|
||||
```
|
||||
|
||||
|
||||
### Linux
|
||||
|
||||
```
|
||||
|
|
@ -137,6 +136,13 @@ KUBERNETES_PUBLIC_ADDRESS=$(aws elb describe-load-balancers \
|
|||
jq -r '.LoadBalancerDescriptions[].DNSName')
|
||||
```
|
||||
|
||||
#### Azure
|
||||
|
||||
```shell
|
||||
KUBERNETES_PUBLIC_ADDRESS=$(az network public-ip show -g kubernetes \
|
||||
-n kubernetes-pip --query "ipAddress" -otsv)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
Create the `kubernetes-csr.json` file:
|
||||
|
|
@ -230,7 +236,7 @@ done
|
|||
The following command will:
|
||||
* Extract the public IP address for each Kubernetes host
|
||||
* Copy the TLS certificates and keys to each Kubernetes host using `scp`
|
||||
|
||||
|
||||
```
|
||||
for host in ${KUBERNETES_HOSTS[*]}; do
|
||||
PUBLIC_IP_ADDRESS=$(aws ec2 describe-instances \
|
||||
|
|
@ -240,3 +246,19 @@ for host in ${KUBERNETES_HOSTS[*]}; do
|
|||
ubuntu@${PUBLIC_IP_ADDRESS}:~/
|
||||
done
|
||||
```
|
||||
|
||||
### Azure
|
||||
|
||||
The following command will:
|
||||
* Extract the public IP address for each Kubernetes host
|
||||
* Copy the TLS certificates and keys to each Kubernetes host using `scp`
|
||||
|
||||
```shell
|
||||
for host in ${KUBERNETES_HOSTS[*]}; do
|
||||
PUBLIC_IP_ADDRESS=$(az network public-ip show -g kubernetes \
|
||||
-n ${host}-pip --query "ipAddress" -otsv)
|
||||
|
||||
scp ca.pem kubernetes-key.pem kubernetes.pem \
|
||||
$(whoami)@${PUBLIC_IP_ADDRESS}:~/
|
||||
done
|
||||
```
|
||||
|
|
@ -108,6 +108,12 @@ INTERNAL_IP=$(curl -s -H "Metadata-Flavor: Google" \
|
|||
INTERNAL_IP=$(curl -s http://169.254.169.254/latest/meta-data/local-ipv4)
|
||||
```
|
||||
|
||||
#### Azure
|
||||
|
||||
```shell
|
||||
INTERNAL_IP=$(ip addr show eth0 | grep -oP '(?<=inet\s)\d+(\.\d+){3}')
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
Each etcd member must have a unique name within an etcd cluster. Set the etcd name:
|
||||
|
|
@ -168,4 +174,4 @@ member 3a57933972cb5131 is healthy: got healthy result from https://10.240.0.12:
|
|||
member f98dc20bce6225a0 is healthy: got healthy result from https://10.240.0.10:2379
|
||||
member ffed16798470cab5 is healthy: got healthy result from https://10.240.0.11:2379
|
||||
cluster is healthy
|
||||
```
|
||||
```
|
||||
|
|
@ -138,6 +138,12 @@ INTERNAL_IP=$(curl -s -H "Metadata-Flavor: Google" \
|
|||
INTERNAL_IP=$(curl -s http://169.254.169.254/latest/meta-data/local-ipv4)
|
||||
```
|
||||
|
||||
#### Azure
|
||||
|
||||
```shell
|
||||
INTERNAL_IP=$(ip addr show eth0 | grep -oP '(?<=inet\s)\d+(\.\d+){3}')
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
Create the systemd unit file:
|
||||
|
|
@ -234,7 +240,6 @@ sed -i s/INTERNAL_IP/$INTERNAL_IP/g kube-controller-manager.service
|
|||
sudo mv kube-controller-manager.service /etc/systemd/system/
|
||||
```
|
||||
|
||||
|
||||
```
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable kube-controller-manager
|
||||
|
|
@ -343,3 +348,26 @@ aws elb register-instances-with-load-balancer \
|
|||
--load-balancer-name kubernetes \
|
||||
--instances ${CONTROLLER_0_INSTANCE_ID} ${CONTROLLER_1_INSTANCE_ID} ${CONTROLLER_2_INSTANCE_ID}
|
||||
```
|
||||
|
||||
### Azure
|
||||
|
||||
```shell
|
||||
az network lb probe create -g kubernetes \
|
||||
-n kubernetes-apiserver-check \
|
||||
--lb-name kubernetes-lb \
|
||||
--protocol http \
|
||||
--port 8080 \
|
||||
--path /healthz
|
||||
```
|
||||
|
||||
```shell
|
||||
az network lb rule create -g kubernetes \
|
||||
-n kubernetes-apiserver-rule \
|
||||
--protocol tcp \
|
||||
--lb-name kubernetes-lb \
|
||||
--frontend-ip-name LoadBalancerFrontEnd \
|
||||
--frontend-port 6443 \
|
||||
--backend-pool-name kubernetes-lb-pool \
|
||||
--backend-port 6443 \
|
||||
--probe-name kubernetes-apiserver-check
|
||||
```
|
||||
|
|
@ -15,7 +15,6 @@ Kubernetes worker nodes are responsible for running your containers. All Kuberne
|
|||
|
||||
Some people would like to run workers and cluster services anywhere in the cluster. This is totally possible, and you'll have to decide what's best for your environment.
|
||||
|
||||
|
||||
## Provision the Kubernetes Worker Nodes
|
||||
|
||||
Run the following commands on `worker0`, `worker1`, `worker2`:
|
||||
|
|
|
|||
|
|
@ -38,6 +38,14 @@ KUBERNETES_PUBLIC_ADDRESS=$(aws elb describe-load-balancers \
|
|||
--load-balancer-name kubernetes | \
|
||||
jq -r '.LoadBalancerDescriptions[].DNSName')
|
||||
```
|
||||
|
||||
#### Azure
|
||||
|
||||
```shell
|
||||
KUBERNETES_PUBLIC_ADDRESS=$(az network public-ip show -g kubernetes \
|
||||
-n kubernetes-pip --query "ipAddress" -otsv)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
Recall the token we setup for the admin user:
|
||||
|
|
@ -97,4 +105,4 @@ NAME STATUS AGE
|
|||
worker0 Ready 7m
|
||||
worker1 Ready 5m
|
||||
worker2 Ready 2m
|
||||
```
|
||||
```
|
||||
|
|
@ -118,3 +118,44 @@ aws ec2 create-route \
|
|||
--destination-cidr-block 10.200.2.0/24 \
|
||||
--instance-id ${WORKER_2_INSTANCE_ID}
|
||||
```
|
||||
|
||||
### Azure
|
||||
|
||||
```shell
|
||||
az network route-table create -g kubernetes \
|
||||
-n kubernetes-routes
|
||||
```
|
||||
|
||||
```shell
|
||||
az network vnet subnet update -g kubernetes \
|
||||
-n kubernetes-subnet \
|
||||
--vnet-name kubernetes-vnet \
|
||||
--route-table kubernetes-routes
|
||||
```
|
||||
|
||||
```shell
|
||||
az network route-table route create -g kubernetes \
|
||||
-n kubernetes-route-10-200-0-0-24 \
|
||||
--route-table-name kubernetes-routes \
|
||||
--address-prefix 10.200.0.0/24 \
|
||||
--next-hop-ip-address 10.240.0.20 \
|
||||
--next-hop-type VirtualAppliance
|
||||
```
|
||||
|
||||
```shell
|
||||
az network route-table route create -g kubernetes \
|
||||
-n kubernetes-route-10-200-1-0-24 \
|
||||
--route-table-name kubernetes-routes \
|
||||
--address-prefix 10.200.1.0/24 \
|
||||
--next-hop-ip-address 10.240.0.21 \
|
||||
--next-hop-type VirtualAppliance
|
||||
```
|
||||
|
||||
```shell
|
||||
az network route-table route create -g kubernetes \
|
||||
-n kubernetes-route-10-200-2-0-24 \
|
||||
--route-table-name kubernetes-routes \
|
||||
--address-prefix 10.200.2.0/24 \
|
||||
--next-hop-ip-address 10.240.0.22 \
|
||||
--next-hop-type VirtualAppliance
|
||||
```
|
||||
|
|
@ -79,6 +79,29 @@ NODE_PUBLIC_IP=$(aws ec2 describe-instances \
|
|||
jq -j '.Reservations[].Instances[].PublicIpAddress')
|
||||
```
|
||||
|
||||
#### Azure
|
||||
|
||||
```shell
|
||||
az network nsg rule create -g kubernetes \
|
||||
-n kubernetes-allow-nginx \
|
||||
--access allow \
|
||||
--destination-address-prefix '*' \
|
||||
--destination-port-range ${NODE_PORT} \
|
||||
--direction inbound \
|
||||
--nsg-name kubernetes-nsg \
|
||||
--protocol tcp \
|
||||
--source-address-prefix '*' \
|
||||
--source-port-range '*' \
|
||||
--priority 1002
|
||||
```
|
||||
|
||||
Grab the `EXTERNAL_IP` for one of the worker nodes:
|
||||
|
||||
```
|
||||
NODE_PUBLIC_IP=$(az network public-ip show -g kubernetes \
|
||||
-n worker0-pip --query "ipAddress" -otsv)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
Test the nginx service using cURL:
|
||||
|
|
|
|||
|
|
@ -205,3 +205,9 @@ DHCP_OPTION_SET_ID=$(aws ec2 describe-dhcp-options \
|
|||
aws ec2 delete-dhcp-options \
|
||||
--dhcp-options-id ${DHCP_OPTION_SET_ID}
|
||||
```
|
||||
|
||||
## Azure
|
||||
|
||||
```shell
|
||||
az group delete -n kubernetes
|
||||
```
|
||||
|
|
|
|||
Loading…
Reference in New Issue