Automating RDS Reader Replica Counts Using AWS Lambda and Boto3

Introduction

Managing RDS read replicas dynamically can improve database performance and reduce costs. This blog will walk you through an AWS Lambda function that adjusts the read replica count for RDS clusters based on custom tags and time-based triggers.

Overview

The Lambda function performs the following actions:

1. Lists all RDS clusters in the account.
2. Retrieves tags for each cluster.
3. Checks for specific tags indicating when to adjust the reader replica count.
4. Dynamically updates the read replica count using the Application Auto Scaling API.

Prerequisites

1. IAM Role: Ensure the Lambda function has sufficient permissions to describe RDS clusters, list tags, and update the scaling targets.
2. Tagged RDS Clusters: Add specific tags to your RDS clusters to indicate when to update reader replicas.
3. Boto3: AWS SDK for Python to interact with AWS services.

Lambda Function Code

import boto3
import os
from datetime import datetime, timezone
# Initialize AWS clients rds_client and appautoscaling_client
# Constants
SERVICE_NAMESPACE = 'rds'
DIMENSION_NAME = 'replica'
RESOURCE_ID_PREFIX = 'cluster:'
def list_all_rds_clusters():
    """
    Fetch all RDS clusters in the account.
    """
    clusters = []
    response = rds_client.describe_db_clusters()
    for cluster in response['DBClusters']:
        clusters.append({
            'DBClusterIdentifier': cluster['DBClusterIdentifier'],
            'DBClusterArn': cluster['DBClusterArn']
        })
    return clusters
def get_cluster_tags(cluster_arn):
    """
    Retrieve tags for a specific RDS cluster.
    """
    response = rds_client.list_tags_for_resource(ResourceName=cluster_arn)
    tags = {tag['Key']: tag['Value'] for tag in response['TagList']}
    return tags
def update_min_readers(cluster_id, min_count, max_count):
    print(f"Updating min reader count for {cluster_id} to {min_count} and max to {max_count}")
    resource_id = f"{RESOURCE_ID_PREFIX}{cluster_id}"
    response = appautoscaling_client.register_scalable_target(
        ServiceNamespace=SERVICE_NAMESPACE,
        ResourceId=resource_id,
        ScalableDimension='rds:cluster:ReadReplicaCount',
        MinCapacity=int(min_count),
        MaxCapacity=int(max_count)
    )
    print(f"Updated successfully: {response}")
def lambda_handler(event, context):
    """
    Lambda handler function.
    """
    try:
        current_time = datetime.now(timezone.utc).strftime("%H:%M")
        print(f"Current Time (UTC): {current_time}")
        clusters = list_all_rds_clusters()
        print(f"Found {len(clusters)} RDS clusters.")
        for cluster in clusters:
            cluster_id = cluster['DBClusterIdentifier']
            cluster_arn = cluster['DBClusterArn']
            print(f"Checking cluster: {cluster_id}")
            tags = get_cluster_tags(cluster_arn)
            print(f"Tags for {cluster_id}: {tags}")
            if tags.get('update_replica_count', 'no').lower() == 'yes':
                print(f"'update_replica_count' is set to 'yes' for {cluster_id}.")
                morning_time = tags.get('morning_time')
                night_time = tags.get('night_time')
                min_readers_morning = tags.get('min_readers_morning')
                min_readers_night = tags.get('min_readers_night')
                max_count = tags.get('max_count')
                if not (morning_time and night_time and min_readers_morning and min_readers_night):
                    print(f"Missing required tags for cluster {cluster_id}. Skipping...")
                    continue
                if current_time == morning_time:
                    update_min_readers(cluster_id, min_readers_morning, max_count)
                elif current_time == night_time:
                    update_min_readers(cluster_id, min_readers_night, max_count)
                else:
                    print(f"No action needed for {cluster_id} at this time.")
            else:
                print(f"'update_replica_count' is not set to 'yes' for {cluster_id}. Skipping...")
        print("RDS reader replica count check completed successfully.")
        return {
            'statusCode': 200,
            'body': 'RDS reader replica count updated successfully where applicable!'
        }
    except Exception as e:
        print(f"Error occurred: {str(e)}")
        return {
            'statusCode': 500,
            'body': f"Error: {str(e)}"
        }

Key Concepts

1. Tags for Time-Based Control:
   update_replica_count: Set to yes to enable updates for the RDS cluster.
   morning_time and night_time: Define when to update reader replicas (e.g., 06:00 and 18:00).
   min_readers_morning and min_readers_night: Number of reader replicas for morning and night periods.
   max_count: Maximum reader replica count.
2. Auto Scaling:
   Uses the register_scalable_target method to set min and max reader counts.
3. Error Handling:
   The function gracefully handles errors and logs useful debug information.

How to Deploy

1. Create Lambda Function
   Open AWS Lambda and create a new function.
   Upload the code or use the in-line editor.
2. Set Environment Variables
3. Add necessary permissions for rds:DescribeDBClusters, rds:ListTagsForResource, and application-autoscaling:RegisterScalableTarget.
4. Test the Function
   Trigger the Lambda function manually to verify it lists clusters, reads tags, and updates replica counts as expected.

Conclusion

With this automated approach, you can ensure optimal resource allocation for your RDS reader replicas. By tagging RDS clusters appropriately, you control when and how the scaling happens. This setup reduces manual effort, optimizes performance, and controls costs.