From add2a072efc051f0dcb035f70fbb02cd2c9d2046 Mon Sep 17 00:00:00 2001 From: Tyler Britten Date: Fri, 21 Oct 2016 10:15:12 -0400 Subject: [PATCH 1/4] Add initial OpenStack Provisiong Guides --- docs/01-infrastructure-os.md | 184 +++++++++++++++++++++++++++++++++++ docs/01-infrastructure.md | 3 +- 2 files changed, 186 insertions(+), 1 deletion(-) create mode 100644 docs/01-infrastructure-os.md diff --git a/docs/01-infrastructure-os.md b/docs/01-infrastructure-os.md new file mode 100644 index 0000000..507c8ca --- /dev/null +++ b/docs/01-infrastructure-os.md @@ -0,0 +1,184 @@ +# Cloud Infrastructure Provisioning - OpenStack + +This lab will walk you through provisioning the compute instances required for running a H/A Kubernetes cluster. A total of 6 virtual machines will be created. + +After completing this guide you should have the following compute instances: + +``` +openstack server list +``` + +```` ++--------------------------------------+-----------------+-----------+----------------------------------------+---------------------+ +| ID | Name | Status | Networks | Image Name | ++--------------------------------------+-----------------+-----------+----------------------------------------+---------------------+ +| 17da9ba7-a0c3-415a-9fe2-b2729d4ba3da | worker2 | ACTIVE | kubernetes=10.240.0.22 | ubuntu-16.04 | +| d52281ba-0a76-4abf-addb-cd56c79d3f1d | worker1 | ACTIVE | kubernetes=10.240.0.21 | ubuntu-16.04 | +| f44c0c77-9810-4cf4-977e-45dafbe87074 | worker0 | ACTIVE | kubernetes=10.240.0.20 | ubuntu-16.04 | +| 96e690b4-e8cb-4733-aa1d-5262106181a2 | controller2 | ACTIVE | kubernetes=10.240.0.12 | ubuntu-16.04 | +| d69f09c1-00e5-465a-831c-446206461d28 | controller1 | ACTIVE | kubernetes=10.240.0.11 | ubuntu-16.04 | +| 80fc744c-d20e-4f24-9b10-c8a26ffbade3 | controller0 | ACTIVE | kubernetes=10.240.0.10, 169.45.x.x | ubuntu-16.04 | ++--------------------------------------+-----------------+-----------+----------------------------------------+---------------------+ +```` + +> All machines will be provisioned with fixed private IP addresses to simplify the bootstrap process. + +To make our Kubernetes control plane remotely accessible, a Floating IP address will be assigned to one of the Kubernetes controllers. You can also assign floating IPs to all the nodes. In this example we're going to assign an IP to controller0 and use that to access the remailing nodes. + +## Networking + +``` +Create a Kubernetes network: + +``` +openstack network create kubernetes +``` + +Create a subnet for the Kubernetes cluster: + +``` +openstack subnet create --network kubernetes \ + --subnet-range 10.240.0.0/24 kubernetes +``` +Create a router for the network: + +``` +openstack router create kubernetes +``` +openstack router add subnet kubernetes kubernetes + + +### Firewall Rules + +First, create a security group: +``` +openstack security group create kubernetes +``` + +``` +openstack security group rule create \ + --ingress \ + --protocol icmp \ + --src-ip 0.0.0.0/0 \ + kubernetes +``` + +``` +openstack security group rule create \ + --ingress --src-group kubernetes --protocol udp kubernetes +``` + +``` +openstack security group rule create \ + --ingress --src-group kubernetes --protocol tcp kubernetes +``` + + +``` +openstack security group rule create \ + --ingress \ + --protocol tcp \ + --dst-port 3389 \ + --src-ip 0.0.0.0/0 \ + kubernetes + +``` + +``` +openstack security group rule create \ + --ingress \ + --protocol tcp \ + --dst-port 22 \ + --src-ip 0.0.0.0/0 \ + kubernetes +``` + +``` +openstack security group rule create \ + --ingress \ + --protocol tcp \ + --dst-port 6443 \ + --src-ip 0.0.0.0/0 \ + kubernetes +``` + + +``` +openstack security group rule list kubernetes +``` + +``` ++--------------------------------------+-------------+-----------+------------+--------------------------------------+ +| ID | IP Protocol | IP Range | Port Range | Remote Security Group | ++--------------------------------------+-------------+-----------+------------+--------------------------------------+ +| 110fc25a-6cc7-409f-9b8f-40be05884203 | None | None | | None | +| 2327d33b-e497-4006-87e3-7991810b1686 | udp | None | | 6f6399ef-b69b-49cb-9f97-8fcad96715bf | +| 2dfe89ce-c167-4f75-89df-a7bc3007336d | icmp | 0.0.0.0/0 | | None | +| 2e175bd1-f885-41de-97af-0787be7fba9e | tcp | 0.0.0.0/0 | 3389:3389 | None | +| 39eaea13-92f5-438b-929c-d7585c84e4b2 | tcp | 0.0.0.0/0 | 22:22 | None | +| 5acea256-84b0-420d-923f-f257fe4e7319 | tcp | 0.0.0.0/0 | 6443:6443 | None | +| b6bc42d0-9f3f-4dcf-a5b1-7196968320d3 | tcp | None | | 6f6399ef-b69b-49cb-9f97-8fcad96715bf | +| d1038338-bf4b-4f25-8c29-a104d74c2803 | None | None | | None | ++--------------------------------------+-------------+-----------+------------+--------------------------------------+ +``` + +## Provision Virtual Machines + +All the VMs in this lab will be provisioned using Ubuntu 16.04 mainly because it runs a newish Linux Kernel that has good support for Docker. + +### Virtual Machines + +#### Kubernetes Controllers + +``` +openstack server create --image ubuntu-16.04 --flavor m1.small \ + --security-group kubernetes --key-name tbritten \ + --nic net-id=1f9ce4ba-2203-4dc2-b411-c0b35ac588c8,v4-fixed-ip=10.240.0.10 \ + controller0 +``` + +``` +openstack server create --image ubuntu-16.04 --flavor m1.small \ + --security-group kubernetes --key-name tbritten \ + --nic net-id=1f9ce4ba-2203-4dc2-b411-c0b35ac588c8,v4-fixed-ip=10.240.0.11 \ + controller1 +``` + +``` +openstack server create --image ubuntu-16.04 --flavor m1.small \ + --security-group kubernetes --key-name tbritten \ + --nic net-id=1f9ce4ba-2203-4dc2-b411-c0b35ac588c8,v4-fixed-ip=10.240.0.12 \ + controller2 +``` + +#### Kubernetes Workers + +``` +openstack server create --image ubuntu-16.04 --flavor m1.small \ + --security-group kubernetes --key-name tbritten \ + --nic net-id=1f9ce4ba-2203-4dc2-b411-c0b35ac588c8,v4-fixed-ip=10.240.0.20 \ + worker0 +``` + +``` +openstack server create --image ubuntu-16.04 --flavor m1.small \ + --security-group kubernetes --key-name tbritten \ + --nic net-id=1f9ce4ba-2203-4dc2-b411-c0b35ac588c8,v4-fixed-ip=10.240.0.21 \ + worker1 +``` + +``` +openstack server create --image ubuntu-16.04 --flavor m1.small \ + --security-group kubernetes --key-name tbritten \ + --nic net-id=1f9ce4ba-2203-4dc2-b411-c0b35ac588c8,v4-fixed-ip=10.240.0.22 \ + worker2 +``` + +### Kubernetes Public Address + +Attached a floating IP to the controller0 to allow for remote access: + +``` +openstack server add floating ip controller0 169.45.x.x +``` + diff --git a/docs/01-infrastructure.md b/docs/01-infrastructure.md index bd4ae69..0e2dab1 100644 --- a/docs/01-infrastructure.md +++ b/docs/01-infrastructure.md @@ -1,8 +1,9 @@ # Cloud Infrastructure Provisioning -Kubernetes can be installed just about anywhere physical or virtual machines can be run. In this lab we are going to focus on [Google Cloud Platform](https://cloud.google.com/) and [Amazon Web Services](https://aws.amazon.com). +Kubernetes can be installed just about anywhere physical or virtual machines can be run. In this lab we are going to focus on [Google Cloud Platform](https://cloud.google.com/), [Amazon Web Services](https://aws.amazon.com), and [OpenStack](https://openstack.org) This lab will walk you through provisioning the compute instances required for running a H/A Kubernetes cluster. * [Cloud Infrastructure Provisioning - Google Cloud Platform](01-infrastructure-gcp.md) * [Cloud Infrastructure Provisioning - Amazon Web Services](01-infrastructure-aws.md) +* [Cloud Infrastructure Provisioning - OpenStack](01-infrastructure-os.md) From ec38f3a97107bef17af21d6979f04ce5cc2b4824 Mon Sep 17 00:00:00 2001 From: Tyler Britten Date: Fri, 21 Oct 2016 10:21:41 -0400 Subject: [PATCH 2/4] Fixed Networking, Formatting --- docs/01-infrastructure-os.md | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/docs/01-infrastructure-os.md b/docs/01-infrastructure-os.md index 507c8ca..86d1fc1 100644 --- a/docs/01-infrastructure-os.md +++ b/docs/01-infrastructure-os.md @@ -27,7 +27,7 @@ To make our Kubernetes control plane remotely accessible, a Floating IP address ## Networking -``` + Create a Kubernetes network: ``` @@ -44,8 +44,19 @@ Create a router for the network: ``` openstack router create kubernetes +``` + +Attach the network to the router: + ``` openstack router add subnet kubernetes kubernetes +``` + +Attack the router to the external network: + +``` +neutron router-gateway-set kubernetes external +``` ### Firewall Rules From 292550c792531a3ea6037dc8802854dc4722bc0a Mon Sep 17 00:00:00 2001 From: Tyler Britten Date: Fri, 21 Oct 2016 13:09:12 -0400 Subject: [PATCH 3/4] Updated the rest of the docs --- docs/02-certificate-authority.md | 33 ++++++++++++++++++++++++++++++++ docs/03-etcd.md | 2 +- docs/04-kubernetes-controller.md | 2 +- docs/06-kubectl.md | 7 +++++++ docs/07-network.md | 15 +++++++++++++++ docs/09-smoke-test.md | 19 ++++++++++++++++++ docs/10-cleanup.md | 29 ++++++++++++++++++++++++++++ 7 files changed, 105 insertions(+), 2 deletions(-) diff --git a/docs/02-certificate-authority.md b/docs/02-certificate-authority.md index c33e51a..ebdce05 100644 --- a/docs/02-certificate-authority.md +++ b/docs/02-certificate-authority.md @@ -137,6 +137,12 @@ KUBERNETES_PUBLIC_ADDRESS=$(aws elb describe-load-balancers \ jq -r '.LoadBalancerDescriptions[].DNSName') ``` +#OpenStack + +``` +KUBERNETES_PUBLIC_ADDRESS=$(openstack server show controller0 -f shell |grep addresses | awk '{print $2}'| sed 's/"$//') +``` + --- Create the `kubernetes-csr.json` file: @@ -239,3 +245,30 @@ for host in ${KUBERNETES_HOSTS[*]}; do ubuntu@${PUBLIC_IP_ADDRESS}:~/ done ``` + +### OpenStack +Since only controller0 has a public IP, you will need to SCP controller0 and then scp it to the remaining 5 hosts from there. + +Copy to controller0: + +``` + scp ca.pem kubernetes-key.pem kubernetes.pem \ + ubuntu@${KUBERNETES_PUBLIC_ADDRESS}:~/ +``` + +Now SSH to controller0. + +Set the list of Kubernetes hosts where the certs should be copied to: + +``` +KUBERNETES_HOSTS=(10.240.0.10 10.240.0.11 10.240.0.12 10.240.0.20 10.240.0.21 10.240.0.22) +``` + +And then copy the TLS certs: + +``` +for host in ${KUBERNETES_HOSTS[*]}; do + scp ca.pem kubernetes-key.pem kubernetes.pem \ + ubuntu@${KUBERNETES_HOSTS}:~/ +done +``` diff --git a/docs/03-etcd.md b/docs/03-etcd.md index f64271a..f84a0d0 100644 --- a/docs/03-etcd.md +++ b/docs/03-etcd.md @@ -101,7 +101,7 @@ INTERNAL_IP=$(curl -s -H "Metadata-Flavor: Google" \ http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/ip) ``` -#### AWS +#### AWS / OpenStack ``` INTERNAL_IP=$(curl -s http://169.254.169.254/latest/meta-data/local-ipv4) diff --git a/docs/04-kubernetes-controller.md b/docs/04-kubernetes-controller.md index 46e31d6..8d62ba0 100644 --- a/docs/04-kubernetes-controller.md +++ b/docs/04-kubernetes-controller.md @@ -132,7 +132,7 @@ INTERNAL_IP=$(curl -s -H "Metadata-Flavor: Google" \ http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/ip) ``` -#### AWS +#### AWS / OpenStack ``` INTERNAL_IP=$(curl -s http://169.254.169.254/latest/meta-data/local-ipv4) diff --git a/docs/06-kubectl.md b/docs/06-kubectl.md index c75b17b..38881c3 100644 --- a/docs/06-kubectl.md +++ b/docs/06-kubectl.md @@ -36,6 +36,13 @@ KUBERNETES_PUBLIC_ADDRESS=$(aws elb describe-load-balancers \ --load-balancer-name kubernetes | \ jq -r '.LoadBalancerDescriptions[].DNSName') ``` + +#OpenStack + +``` +KUBERNETES_PUBLIC_ADDRESS=$(openstack server show controller0 -f shell |grep addresses | awk '{print $2}'| sed 's/"$//') +``` + --- Recall the token we setup for the admin user: diff --git a/docs/07-network.md b/docs/07-network.md index 5b99809..cbd73f9 100644 --- a/docs/07-network.md +++ b/docs/07-network.md @@ -118,3 +118,18 @@ aws ec2 create-route \ --destination-cidr-block 10.200.2.0/24 \ --instance-id ${WORKER_2_INSTANCE_ID} ``` + +### OpenStack + +``` +openstack router set --route destination=10.200.0.0/24,gateway=10.240.0.20 kubernetes +``` + +``` +openstack router set --route destination=10.200.1.0/24,gateway=10.240.0.21 kubernetes +``` + +``` +openstack router set --route destination=10.200.2.0/24,gateway=10.240.0.22 kubernetes +``` + diff --git a/docs/09-smoke-test.md b/docs/09-smoke-test.md index a812108..2edc02b 100644 --- a/docs/09-smoke-test.md +++ b/docs/09-smoke-test.md @@ -79,6 +79,25 @@ NODE_PUBLIC_IP=$(aws ec2 describe-instances \ jq -j '.Reservations[].Instances[].PublicIpAddress') ``` +#### OpenStack + +``` +openstack security group rule create \ + --ingress \ + --protocol tcp \ + --dst-port ${NODE_PORT} \ + --src-ip 0.0.0.0/0 \ + kubernetes + +``` + +Add an `EXTERNAL_IP` for one of the worker nodes: + +``` +NODE_PUBLIC_IP=169.45.x.x +openstack server add floating ip worker0 ${NODE_PUBLIC_IP} +``` + --- Test the nginx service using cURL: diff --git a/docs/10-cleanup.md b/docs/10-cleanup.md index 452b58a..8323786 100644 --- a/docs/10-cleanup.md +++ b/docs/10-cleanup.md @@ -205,3 +205,32 @@ DHCP_OPTION_SET_ID=$(aws ec2 describe-dhcp-options \ aws ec2 delete-dhcp-options \ --dhcp-options-id ${DHCP_OPTION_SET_ID} ``` + +## OpenStack + +### Virtual Machines + +``` +openstack server delete \ + controller0 controller1 controller2 \ + worker0 worker1 worker2 +``` + +### Networking + +``` + openstack security group delete kubernetes +``` + +``` +openstack subnet delete kubernetes +``` + +``` +openstack network delete kubernetes +``` + +``` +openstack router delete kubernetes +``` + From e66b595e2e1d7cf1fde488a52439e0d472f46a82 Mon Sep 17 00:00:00 2001 From: Tyler Britten Date: Fri, 21 Oct 2016 13:16:36 -0400 Subject: [PATCH 4/4] Added info to readme for OpenStack --- README.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/README.md b/README.md index fc07bb9..95e5b2b 100644 --- a/README.md +++ b/README.md @@ -6,6 +6,7 @@ This tutorial is optimized for learning, which means taking the long route to he * [Google Compute Engine](https://cloud.google.com/compute) * [Amazon EC2](https://aws.amazon.com/ec2) +* [OpenStack](https://openstack.org) > The results of this tutorial should not be viewed as production ready, and may receive limited support from the community, but don't let that prevent you from learning! @@ -47,12 +48,15 @@ AWS * The us-west-2 region will be used + + ## Platforms This tutorial assumes you have access to one of the following: * [Google Cloud Platform](https://cloud.google.com) and the [Google Cloud SDK](https://cloud.google.com/sdk/) (125.0.0+) * [Amazon Web Services](https://aws.amazon.com), the [AWS CLI](https://aws.amazon.com/cli) (1.10.63+), and [jq](https://stedolan.github.io/jq) (1.5+) +* [OpenStack](https://openstack.org) and the [Python OpenStack Client](https://github.com/openstack/python-openstackclient) ## Labs