Setting up EC2 health check alerts using Lambda, CloudWatch, and SNS

Setting up EC2 health check alerts using Lambda, CloudWatch, and SNS

Amazon Elastic Compute Cloud (EC2) instances play a pivotal role worldwide, being utilized by millions of users. They serve as the backbone for hosting critical applications and services, while ensuring their robustness and accessibility. Imagine a system where potential issues are identified proactively, downtime is minimized, and operational efficiency is optimized. Health checks enable Amazon EC2 Auto Scaling to identify unhealthy instances and terminate them.

This blog explains the process of setting up EC2 health check alerts using Lambda, CloudWatch, and SNS — empowering you to take control of your instances’ well-being.

Why automated health checks?

Before diving into the technical details, it’s crucial to understand how why automated health checks for your EC2 instances are essential. Here are the benefits:

1. Proactive issue detection: Automated health checks can detect issues in real-time or at predefined intervals, allowing you to take corrective actions before they affect your applications.
2. Cost reduction: By automating alarms, you can prevent prolonged downtime, which can lead to significant revenue loss or increased operational costs.
3. Efficiency boost: Manual health checks are time-consuming and error prone. Automation frees up your team to focus on more critical tasks.
4. Improved reliability: Automating health checks ensures consistency and reliability. Human errors can be reduced significantly.

Usually the objective is to establish an alert system whenever an EC2 instance encounters a health status failure within the AWS account. This system should be configured to send email notifications to a specified email address on failure. This proactive setup ensures timely awareness and facilitates a prompt response to any issues affecting the health of EC2 instances across the entire AWS account.

We attempted to configure this alert system using different methods, encountering challenges along the way:

1. AWS-CLI: We ensured that AWS CLI installed and configured with the necessary IAM permissions to create alarms. But even when the alarm was properly configured, it was stuck in “insufficient state”. So, we moved with the next solution.
2. Using AWS Cloud-Watch alarms (metric wise): If there are a smaller number of instances, an alarm can be set for each instance with the desired metric, but for greater number of instances, it is not a feasible solution.
3. Configure Lambda function: The above two solutions were not working in our case as the number of instances were high. So, we opted to use an automation or scripting approach to dynamically create alarms for each instance.

Here’s a high-level outline of how you could achieve this:

Automating EC2 health check alarms with AWS Lambda

In today’s cloud-driven world, managing infrastructure at scale is a challenging task. One of the most important aspects of managing cloud resources is ensuring their health and availability. Amazon Web Services (AWS) offers a range of services to simplify this challenge. AWS Lambda, a serverless compute service, is a powerful tool for automating various tasks in AWS environments. This is how you can use AWS Lambda to automate health check alarms for EC2 instances in the same region.

Prerequisites of AWS Lambda

Ensure you have the following:

1. An AWS account.
2. EC2 instances that you want to monitor.
3. Basic knowledge of AWS services like Lambda, SNS (Simple Notification Service), and CloudWatch.

Step 1: Create an SNS Topic

The first step is to set up a Simple Notification Service (SNS) topic that will be used to send notifications when alarms are triggered.

1. Go to the AWS Management Console and navigate to the SNS service.
2. Click on “Create Topic” and provide a name and display name for your topic. You can leave the other settings as default.
3. Once the topic is created, note down its ARN (Amazon Resource Name). You will need this when configuring your Lambda function.

Step 2: Create an IAM Role for Lambda

To allow your Lambda function to interact with EC2 instances and publish messages to the SNS topic, you need to create an IAM (Identity and Access Management) role with the necessary permissions.

1. Go to the AWS Management Console and navigate to the IAM service.
2. Click on “Roles” in the left-hand menu and then click “Create Role.”
3. For the use case, select “Lambda” as the service that will use this role, then click “Next: Permissions.”
4. In the permissions search box, type “AmazonEC2ReadOnlyAccess” to attach the read-only EC2 permissions policy. This policy allows the Lambda function to describe EC2 instances.
5. Additionally, you will need to attach the policy “AmazonSNSFullAccess” to allow the Lambda function to publish messages to the SNS topic.
6. Complete the role creation process by specifying a name for your role and adding any necessary tags. Finally, click “Create Role.”

Step 3: Write the Lambda Function

Now, it’s time to write the Lambda function that will perform the health checks and trigger alarms when necessary. (Read more…)