Automating EC2 Instances Start/Stop with Terraform, Lambda, and Python

AWS Lambda is a serverless computing service provided by Amazon Web Services (AWS). It allows you to run your code without provisioning or managing servers. With Lambda, you can execute your code in response to events, such as changes to data in an S3 bucket, updates to a DynamoDB table, or even an HTTP request.

One common use case of AWS Lambda is automating the start and stop of Amazon EC2 instances. EC2 instances are virtual servers in the cloud that provide computing resources for various applications. By automating the start and stop process, you can optimize costs by running instances only when they are needed.

Here’s how the process works:

You write your code logic, such as the code snippet you provided earlier, to perform actions like retrieving instance IDs, checking their state, and starting or stopping them.
You package your code and dependencies into a deployment package, typically a ZIP file.
You create an AWS Lambda function and upload the deployment package to it.
You configure the Lambda function’s trigger to determine when it should execute. For example, you can schedule the function to run at specific times using CloudWatch Events or trigger it manually through API Gateway.
When the Lambda function is triggered, it executes the code within the lambda_handler function.
The code interacts with the AWS SDKs to communicate with the EC2 service, retrieve instance information, and perform the start or stop actions on the instances.
The results of the Lambda function’s execution, such as success or error messages, are logged and can be monitored using CloudWatch Logs.

Here’s how the process works:

You write your code logic, such as the code snippet you provided earlier, to perform actions like retrieving instance IDs, checking their state, and starting or stopping them.
You package your code and dependencies into a deployment package, typically a ZIP file.
You create an AWS Lambda function and upload the deployment package to it.
You configure the Lambda function’s trigger to determine when it should execute. For example, you can schedule the function to run at specific times using CloudWatch Events or trigger it manually through API Gateway.
When the Lambda function is triggered, it executes the code within the lambda_handler function.
The code interacts with the AWS SDKs to communicate with the EC2 service, retrieve instance information, and perform the start or stop actions on the instances.
The results of the Lambda function’s execution, such as success or error messages, are logged and can be monitored using CloudWatch Logs.

By leveraging AWS Lambda, you can automate the start and stop process, eliminating the need for manual intervention. This helps optimize costs by running instances only when necessary, such as during business hours or during specific time periods.

It’s worth mentioning that AWS Lambda offers many other capabilities beyond EC2 instance management. It supports various programming languages, provides scalability and fault tolerance out of the box, and integrates with other AWS services for building serverless applications.

I hope this provides you with a good understanding of AWS Lambda and its usage in automating the start and stop of EC2 instances.

import boto3
import os
import json

def lambda_handler(event, context):
try:
tags = os.environ["Tags_json"]
dictionary = json.loads(tags) # Convert str to Dictionary using Json
print(dictionary)
print(type(dictionary))

ec2 = boto3.client('ec2')

instance_ids = [] # Placeholder for instance ids

for (key, value) in dictionary.items():
print(f'Key: {key}')
print(f'Value: {value}')

NextToken = '' # Placeholder for NextToken

# Describe instances with the specified tag key, value, and stopped state
response = ec2.describe_instances(
Filters=[
{'Name': 'tag-key', 'Values': [key]},
{'Name': 'tag-value', 'Values': [value]},
{'Name': 'instance-state-name', 'Values': ['stopped']}
],
MaxResults=20
)
print(f'EC2 InstanceIds Response: {response}')

# This loop fetches all the instance ids using pagination with NextToken
while True:
for reservation in response['Reservations']:
for instance in reservation['Instances']:
instance_ids.append(instance['InstanceId'])
print(f'Number of instances: {len(instance_ids)}')
print(f'InstanceIds : {instance_ids}')

# If NextToken is present, it retrieves the next set of instances
if 'NextToken' in response:
print("Testing NextToken")
response = ec2.describe_instances(
Filters=[
{'Name': 'tag-key', 'Values': [key]},
{'Name': 'tag-value', 'Values': [value]},
{'Name': 'instance-state-name', 'Values': ['stopped']}
],
NextToken=response['NextToken']
)
print(f'Response: {response}')
print(f'NextToken: {NextToken}')
else:
break

if instance_ids:
# Start the instances
ec2.start_instances(InstanceIds=instance_ids)
print("EC2 instances started: {}".format(instance_ids))
else:
print("No EC2 instances matching the filter.")

except Exception as e:
print("An error occurred:", str(e))

The code snippet you provided is designed to be used within an AWS Lambda function. When executed, the code performs the following actions:

It imports the necessary libraries: boto3 for interacting with AWS services, os for accessing environment variables, and json for working with JSON data.
The lambda_handler function is the entry point for the Lambda function. It takes in two parameters: event and context. These parameters provide information about the triggering event and the execution context.
Inside the lambda_handler, the code retrieves the environment variable Tags_json using os.environ. This variable is expected to contain a JSON string representing a dictionary of tags.
The code then uses json.loads to convert the JSON string into a dictionary. This dictionary represents the tags and their corresponding values that will be used to filter the EC2 instances.
The code initializes the ec2 client from boto3 to interact with the EC2 service.
A list instance_ids is created as a placeholder for storing the IDs of instances that match the specified tags and are in a stopped state.
The code iterates over the key-value pairs in the dictionary using a for loop. For each pair, it performs the following actions:

Prints the key and value.
Sets the NextToken variable to an empty string as a placeholder.
Calls ec2.describe_instances with filters based on the tag key, tag value, and instance state (stopped). This retrieves information about the instances that match the filters.
Prints the response and the number of instances found.
Checks if a NextToken is present in the response. If so, it retrieves the next set of instances by making another describe_instances call with the NextToken parameter. This process continues until all instances have been retrieved.

8. If instance_ids is not empty, it means there are instances that match the filters. The code calls ec2.start_instances with the InstanceIds parameter set to the list of instance IDs. This starts the instances.

9. If instance_ids is empty, it means there are no instances that match the filters.

10. Any exceptions that occur during the execution are caught by the except block, and the error message is printed.

When you deploy this code as an AWS Lambda function, you can configure it to be triggered based on specific events, such as a schedule or manual invocation. Once triggered, the function will execute and perform the desired actions of starting the EC2 instances based on the specified tags and their current state.

To make it more convenient for you, I have created Terraform code that sets up the infrastructure required for the stop/start Lambda function. You can find the code in the following Git repository:

GitHub Repository: EC2-Instances-Start-Stop-with-Terraform-Lambda

The Terraform code in this repository will help you create the necessary resources, including the Lambda function, IAM roles, and any other dependencies required for automating the start and stop of EC2 instances.

Feel free to explore the repository and utilize the Terraform code to set up the environment for your use case.

Automating EC2 Instances Start/Stop with Terraform, Lambda, and Python

Comments

Leave a comment Cancel reply