How to configure Jenkins with High Availability?

In our quest for faster application development, we have invented, tested, and implemented several practices that have revolutionized the way we develop applications. Continuous Integration (CI) is one such DevOps practice, which has augmented the speed of application development by coupling skills of developers with an arsenal of tools. Jenkins is a popular CI tool used to automate complex tasks. As the infrastructure grows, you have to start thinking about securing and load balancing CI/CD tools with Jenkins, which is the heart of any DevOps culture.

Amidst all the comforts it brings to the DevOps pipeline, a single point of failure in Jenkins can break down your whole DevOps process. Hence, it is important to ensure that your Jenkins is fool-proof. Now there are multiple ways to achieve this. You can use enterprise Jenkins offerings but it will add to the cost. For me the simplest one that I would recommend is configuring Jenkins High Availability with two Jenkins masters in which one will always be active to serve builds and another will be passive. The passive or inactive master will get activated if the master that was active earlier goes inactive. Want to know how to set this up? I have got you covered. Before going through the actual setup and configuration, let’s have a look at the prerequisites for this setup –

• AWS Account with access to EC2 and EFS
• Three instances — one for HA proxy and two with Jenkins installed
• Knowledge of EFS (Elastic File System) on AWS
• Knowledge of HAProxy.

Now, before jumping onto the actual setup process let’s walk through some basic concepts that we will be using further –

What is High Availability (HA)?

When we say high availability, we are referring to systems that can operate continuously without failure for a long time. The term implies that the system has been tested thoroughly to stand any sort of failure. Jenkins is a crucial component of DevOps and its downtime may have adverse effects on the DevOps environment. To overcome these, we need a high availability setup for Jenkins.

How HAProxy works?

• The user’s web browser sends a request to the load balancer over port 80 which is assigned to commonly used HTTP, using the fully qualified domain name such as http://jenkins.domain.com
• The Domain Name System (DNS) reaches the IP address of the domain name entered and fetches data from there.
• If a load-balancer server is unavailable, the request to that server will be timed out. If the load-balancer is available, then according to the availability of the machine, it routes requests.
• Referencing the load-balancer pool in its configuration file, HAProxy determines the application server to which the client request has to be routed. This server receiving the request is usually a part of the auto-scaling array comprising dedicated application servers.
• HAProxy forwards this request to the server port referenced in its configuration file (generally port 80).

What is Elastic File System (EFS)?

The EFS service on AWS is similar to the NFS service on Unix systems wherein the mount point is created on network topology and can be shared with multiple systems at the same time. We are going to configure EFS with Jenkins master machines so that both masters can store and access Jenkins jobs from EFS.

Jenkins HA Setup

Our Jenkins HA setup will look like this –

Now let’s see how we are going to configure HAProxy.

As shown in the diagram above, the first tier is the HAProxy. Whenever there is an incoming request for the build, it will first reach the HAProxy. Then, it will decide the machine to which the request can be routed. Here, one of the masters will be in the active state to serve requests and the other one will be passive. Whenever there is a problem with the active master and it goes down, the other master will become active and requests will resume. These requests will then be served by the master that has become active. HA proxy communicates with nodes mainly with the help of tokens that are called heartbeats on active and passive links.

The second tier is Jenkins master setup wherein both masters have the same configuration and share Jenkins home for high availability. So, the project created on one Jenkins master will be available for the second master as well once the configuration is reloaded. For reloading the configuration on the second master, type “/reload” at the end of the Jenkins URL and then hit enter.
E.g. http://myjenkinsserver.com:8080/reload will reload configuration without restarting.

The third tier is the EFS mount point on …read more