How to Identify RDS Reader and Writer Instances Programmatically Using AWS SDK

Managing Amazon RDS clusters involves several critical tasks, including identifying Reader and Writer instances. This information is essential for tasks like scaling Reader instances, modifying instance types, or implementing failover mechanisms. If you’ve worked with RDS clusters, you might have noticed that identifying the role (Reader or Writer) of instances isn’t directly available in some APIs. In this blog, we’ll walk through how to programmatically determine the role of RDS instances in a cluster using the AWS SDK.

The Problem: Determining Reader and Writer Roles in RDS Clusters

RDS clusters consist of a primary Writer instance and one or more Reader instances (handling read-only queries). While the AWS Management Console conveniently labels instances as Readers or Writers, fetching this data programmatically requires additional steps.

The challenge lies in identifying roles because the describe_db_instances API doesn't directly specify whether an instance is a Reader or Writer. However, by combining data from the describe_db_clusters API, we can accurately determine roles.

The Solution

We use the WriterEndpoint attribute from the describe_db_clusters API to identify the Writer instance. The endpoint of each instance, fetched via describe_db_instances, is then compared against the WriterEndpoint to determine its role.

Here’s how you can implement this:

Step-by-Step Implementation

Below is the Python code that accomplishes the task:

import boto3

# Initialize AWS clients
rds_client = boto3.client('rds')

def list_all_rds_clusters():
    clusters = []
    response = rds_client.describe_db_clusters()
    for cluster in response['DBClusters']:
        clusters.append({
            'DBClusterIdentifier': cluster['DBClusterIdentifier'],
            'DBClusterArn': cluster['DBClusterArn']
        })
    return clusters

def list_instances_in_cluster(cluster_id):
    try:
        # Fetch cluster information to get WriterEndpoint
        cluster_response = rds_client.describe_db_clusters(DBClusterIdentifier=cluster_id)
        writer_endpoint = cluster_response['DBClusters'][0].get('WriterEndpoint')

        # Fetch all DB instances and filter those in the cluster
        response = rds_client.describe_db_instances()
        instances = []
        for instance in response['DBInstances']:
            if instance.get('DBClusterIdentifier') == cluster_id:
                endpoint = instance['Endpoint']['Address']
                role = 'WRITER' if endpoint == writer_endpoint else 'READER'
                instances.append({
                    'DBInstanceIdentifier': instance['DBInstanceIdentifier'],
                    'DBInstanceClass': instance['DBInstanceClass'],
                    'InstanceRole': role,
                    'Endpoint': endpoint
                })
        return instances

    except Exception as e:
        print(f"Error listing instances for cluster {cluster_id}: {str(e)}")
        return []

def lambda_handler(event, context):
    clusters = list_all_rds_clusters()
    for cluster in clusters:
        cluster_id = cluster['DBClusterIdentifier']
        print(f"Cluster ID: {cluster_id}")
        instances = list_instances_in_cluster(cluster_id)
        for instance in instances:
            print(f"Instance ID: {instance['DBInstanceIdentifier']}, Role: {instance['InstanceRole']}, Endpoint: {instance['Endpoint']}")

Understanding the Code

1. Fetch All RDS Clusters

The list_all_rds_clusters function retrieves all RDS clusters using describe_db_clusters.

2. Retrieve WriterEndpoint

The WriterEndpoint is extracted from the cluster’s details. This endpoint is unique to the Writer instance.

3. List Instances and Determine Roles

The list_instances_in_cluster function compares each instance’s endpoint against the WriterEndpoint. Instances with matching endpoints are labeled as Writers; others are Readers.

4. Lambda Integration

The lambda_handler function ties everything together, making it easy to deploy this logic as an AWS Lambda function.

Sample Output

When executed, the script produces the following output:

Cluster ID: prod-cluster
Instance ID: prod-cluster-instance-1, Role: READER, Endpoint: prod-cluster-instance-1.example.com
Instance ID: prod-cluster-instance-2, Role: WRITER, Endpoint: prod-cluster-instance-2.example.com
Instance ID: prod-cluster-instance-3, Role: READER, Endpoint: prod-cluster-instance-3.example.com

Use Cases

1. Scaling Reader Instances
   Dynamically identify and modify Reader instances based on workload requirements.
2. Failover and DR
   Automate failover processes by targeting the Writer instance during emergencies.
3. Tag-Based Automation
   Combine this logic with tags to perform instance-type upgrades or other cluster-wide operations.

Conclusion

Identifying RDS instance roles programmatically might seem tricky at first, but combining the WriterEndpoint from describe_db_clusters with the endpoints from describe_db_instances makes it straightforward. This approach is invaluable for automating RDS cluster management tasks.