Provision & Configure AWS ELB to distribute traffic to backend AWS EC2 Instances using Ansible
In order to achieve high availability, at least 2 backend servers should be present and the load balancer will ensure that if 1 backend is not functioning, traffic is directed to the other. ELB is a service provided by AWS that helps to distribute the traffic and provides a single endpoint, where the client can connect. This is very useful when we want to do rolling updates as a Load Balancer will prevent downtime on the client-side.
This article is a continuation of the previous one! So go through that first if you haven’t.
In the previous article, I had explained in detail how to provision multiple EC2 Instances and configure them as Web Servers using Ansible.
At the end of this article, 3 instances were launched.
But there are 2 issues in this setup, especially when it is over the cloud:
- The instances are launched in the same subnet. This means all the instances are launched in the same Availability Zones. If one complete AZ i.e. the data center goes down, all instances will be gone. So there will be no point to launch multiple instances.
This means when launching multiple instances to support High Availability, the instances should be launched in different AZ i.e. in different subnets.
I will show:
how to launch multiple EC2 instances in different subnets
- Even after launching multiple instances, we should provide the client with only 1 IP. The server should have the intelligence to direct the traffic to balance the load. This is done by a load balancer, and I will show:
how to Provision an AWS Elastic Load Balancer
how to add the launched instances as backend servers in the Load Balancer.
Ansible is a handy, fast tool that allows us to write playbooks to help IT Engineers in provisioning. I have created a setup on AWS Cloud using 1 single Ansible Playbook. The steps followed are:
First Play
- Create a Vault to store the AWS Secret & Access Key.
- Create the required variables.
- Set the value of Subnet ID & number of instances.
- Create a Security Group for AWS EC2 instance.
- Provision EC2 Instances on AWS Cloud.
- Then put the Public IP of all the EC2 Instance in a host group “webserver” in the Inventory of Ansible.
Second Play
- This play works on the host group “webserver” and calls the Role I have created to configure Web Server (httpd) on a Linux Instances.
- The Role will install httpd software on all Linux Instances.
- It will copy a dummy code from my GitHub, into the folder in the instances.
- Then Start the httpd service.
Third Play
- First, the vars_file(AWS Credentials) and the variable values are imported.
- Then the Elastic Load Balancer is created.
- Then previously created EC2 Instances are registered as targets in the ELB using “for loop” in Ansible.
In this article, I will be focussing on some editting on First Play & the Third Play. Rest is explained in a very detailed manner in the first part of this article.
Let’s start by building the code:
I have Ansible installed on my local Linux VM — RHEL 8. I am creating the playbook in this VM.
Step 0— Ansible Config File & Ansible Vault
Ansible Config File
Ansible Vault
ansible-vault create mycred.yml“All the variables are well — explained here.”
Step 1— Main Playbook
Play 1 — Configure Localhost for Provisioning AWS Instance
Variables -
- hosts: localhost
gather_facts: no
vars_files:
- mycred.ymlvars:
myport: 81
region: ap-south-1
subnet:
- subnet-c48ee588
- subnet-ca87bda2
sg: websg
type: t2.micro
number: 1
key: key1
id: []
ip: []
🔴 The main thing to note here is the subnet ID. Here you need to specify YOUR Subnet IDs in which YOU want to launch the instances. 🔴
- id & ip
These 2 empty lists are created to be used later. - ip list is required to configure the instances after the launch. We need to tell all the IPs present in different subnets to the second play that will configure these instances with httpd software and test code.
- id list is created to collect all the instance IDs to provide to the third play in which a load balancer will be configured in front of these backend servers. The load balancer will require all the IDs of the instances to connect to them.
Installations -
tasks:
- name: installing python
package:
name: python36
state: present
- name: installing boto3
pip:
name: boto3
state: presentBoto3 is the Amazon Web Services (AWS) SDK for Python. It enables Python developers to create, configure, and manage AWS services.
So using these 2 tasks I am checking the presence of Python & Boto3. If not present these 2 tasks will download the same.
Security Group for EC2 Instance -
- name: create security group
ec2_group:
name: "{{ sg }}"
description: The webservers security group
region: "{{ region }}"
aws_access_key: "{{ access_key }}"
aws_secret_key: "{{ secret_key }}"
rules:
- proto: tcp
from_port: 22
to_port: 22
cidr_ip: 0.0.0.0/0
- proto: tcp
from_port: 80
to_port: 80
cidr_ip: 0.0.0.0/0
- proto: tcp
from_port: "{{ myport }}"
to_port: "{{ myport }}"
cidr_ip: 0.0.0.0/0
rules_egress:
- proto: all
cidr_ip: 0.0.0.0/0Next, I have created the Security Group to be used by the AWS EC2 Instance. It picks up the values from the variables declared above.
In the security group, 2 things have to be set: Ingress and Egress.
Ingress means the traffic that is coming into our website. We need to specify this, keeping in mind what ports we want to keep open. I have kept open 2 ports: SSH, and HTTP.
- SSH so that Ansible can connect to it to do the configuration.
- HTTP so that traffic can hit on the website. The port is specified from the variable provided above.
Egress has been set to all ports so that outbound traffic originating from within a network can go outside to the Public World.
Provision EC2 Instance -
- name: launching ec2 instance
loop: "{{ subnet }}"
ec2:
key_name: "{{ key }}"
instance_type: "{{ type }}"
image: ami-0ebc1ac48dfd14136
wait: true
group: "{{ sg }}"
count: "{{ number }}"
vpc_subnet_id: "{{ item }}"
assign_public_ip: yes
region: "{{ region }}"
state: present
aws_access_key: "{{ access_key }}"
aws_secret_key: "{{ secret_key }}"
instance_tags:
Name: webserver
register: ec2All the variables used are similar to the previous one with only one update. Since I am launching the EC2 instances in different subnets(which are specified earlier) so I am running a loop to create the required number of instances in each subnet.
The whole thing is registered in a variable “ec2” which will be used later.
Adding Instance Public IP & Instance ID to lists -
- set_fact:
id: "{{ id + item['instance_ids'] }}"
with_items:
- "{{ ec2.results }}"
- debug:
msg: "{{ id }}"
- set_fact:
ip: "{{ ip + item['instances'] }}"
with_items:
- "{{ ec2.results }}"
- debug:
msg: "{{ item.public_ip }}"
with_items: "{{ ip }}"Using the module set_fact I am appending the lists. The IP & Instance ID is coming directly from ec2 variable which was stored while creating the instances.
Add to Inventory dynamically -
- name: Add new instance to host group
add_host:
hostname: "{{ item.public_ip }}"
groupname: webserver
loop: "{{ ec2.instances }}"
- name: Wait for SSH to come up
wait_for:
host: "{{ item.public_dns_name }}"
port: 22
state: started
loop: "{{ ec2.instances }}"add_host is a module that helps to add IPs in the inventory file dynamically for use by Ansible. For this, I have used the concept of “for loop” because this playbook can launch as many instances as the user wants.
So in the loop, the variable is ec2.instances & in the hostname keyword, the variable used is item.public_ip. item is a pre-defined variable for the for loops in Ansible.
Since it will take some time to connect to the EC2 Instance, we have to tell Ansible to wait.
wait_for module is used where I have specified the port 22 i.e. SSH and again looped over so that it can let Ansible wait to get connected to all the instances. Once it is connected it goes on to the next task.
Play 2 — Configure the EC2 Instances to work as Web Servers
- hosts: webserver
gather_facts: no
tasks:
- command: curl http://ipv4.icanhazip.com
register: x
- debug:
var: x.stdout
- name: Pass variables to role
include_role:
name: httpdserver
vars:
my_ip: x.stdoutThis part remains the same as before in which I am using include_role module to include the httpdserver role that I have created. It will configure all the running instances as webservers and load a test code in them.
What is a role, how I have created my own role, and how to use it is well explained in the first part of this article.
Tagging again for your ready reference.
Play 2 — Creating the Load Balancer on AWS (AWS ELB)
- hosts: localhost
gather_facts: no
vars_files:
- mycred.ymlvars:
region: ap-south-1
sg: websg
myport: 81
Here I have used the same credential file mycred.yml
It contains the credentials to use the AWS Account.
Then a few variables have been specified to be used while creating the load balancer:
- region
The region in which you want to create the load balancer - sg
It is the same security group that was used earlier. It is used to specify the port on which the load balancer will work. - myport
This variable is the port on which I am creating the load Balancer.
Creating the Load Balancer -
tasks:
- name: create Load Balancer AWS
ec2_elb_lb:
name: ec2-lb
state: present
aws_access_key: "{{ access_key }}"
aws_secret_key: "{{ secret_key }}"
region: "{{ region }}"
cross_az_load_balancing: yes
zones:
- "{{ region }}a"
- "{{ region }}b"
- "{{ region }}c"
scheme: internet-facing
security_group_names: "{{ sg }}"
listeners:
- protocol: http
load_balancer_port: 80
instance_port: "{{ myport }}"
health_check:
ping_protocol: http
ping_port: "{{ myport }}"
ping_path: "/index.php"
response_timeout: 5
interval: 30
unhealthy_threshold: 2
healthy_threshold: 10To create a Load Balancer in AWS using Ansible, I have used the module: ec2_elb_lb. It has the following parameters:
- name
A name has to be specified for the Load Balancer. - aws_access_key & aws_secret_key
These are provided in the Ansible Vault. - region
The region in which you want to create the load balancer. - cross_az_load_balancing
Put “yes” if you want to balance load across multiple AZ. - zones
If the previous option is enabled then, specify the name of zones to be included in the Load Balancer. - scheme
What type of Load Balancer you require: internal or internet-facing. - security_group_names
The name of the security group present in front of the load balancer. - listeners
The port on which the Load Balancer is listening. - health_check
We can enable a health check in the Load Balancer, that will check for a test file in the backend servers within the given response time. If the page responds with status code 200 then only Load Balancer will add it to the backend servers.
- name: add webservers to AWS ELB
local_action: ec2_elb
args:
aws_access_key: "{{ access_key }}"
aws_secret_key: "{{ secret_key }}"
instance_id: "{{ item }}"
ec2_elbs: ec2-lb
state: present
region: "{{ region }}"
loop: "{{ id }}"Finally, we have to add the webservers (configured EC2 Instances) to the ELB. I have put the Instance IDs in a loop to be put in the ELB as backend servers. Some values are used as args that are required while creating ELB.
You can find these code files on my GitHub profile.
Now I will run the playbook -
NOTE: subnet is a required variable and it is mandatory to be specified in the main file first.
ansible-playbook ec2_v2.yml -e number=<number of instances to launch> --ask-vault-pass
