About SRGTechnical articleRecruitment Information
About SRGTechnical articleRecruitment Information
HOME/Technical article/Detailed Explanation of IAM Authentication for AWS ElastiCache & RDS

Detailed Explanation of IAM Authentication for AWS ElastiCache & RDS

Ishikawa Kumo, Service Reliability Group (SRG), Media Management Division@ishikawa_kumo)is.
#SRGThe Service Reliability Group primarily provides comprehensive support for the infrastructure surrounding our media services, focusing on improving existing services, launching new ones, and contributing to open-source software (OSS).
This article outlines the actual implementation steps and common pitfalls encountered during operation of IAM authentication in AWS ElastiCache and RDS. We hope this will be helpful in reducing the operational burden associated with password management and building a more secure and efficient infrastructure.

About ElastiCache & RDS IAM Authentication

With AWS managed data stores, ElastiCache and RDS, the default user authentication for ElastiCache and password authentication by the Master user for RDS have traditionally been the most common methods of authentication.
However, password management has clear limitations.
  • The storage locations for the secrets are scattered.
  • Rotation management is complicated.
  • Issuing permanent credentials for a temporary connection
  • This is a common cause of security incidents.
If you can no longer tolerate these problems, the realistic options areIAM AuthenticationThat's right. By using IAM authentication, you can eliminate the very premise of "having a password."

Use cases

1. Database and cache connections in a container environment

In a container environment,Secret management is the most time-consuming and most prone to errors.Therefore, while direct access from Secrets Manager is desirable from a risk reduction perspective, in practice, a multi-layered design like the following is necessary.
  • Secrets Manager
  • How to inject into a Pod
  • Rotation design
  • IAM permission design
In particular, with EKS, the operation of External Secrets and Secret Operators is required, resulting in high costs for both initial setup and maintenance. Using IAM authentication,
  • Do not store the password in an environment variable.
  • Do not manage as a secret.
  • Connect using only the IAM permissions granted to the Pod.
This configuration becomes possible.

2. DB Client for Local Environment and EC2

Storing passwords in your local environment or on EC2 for temporary connections is inherently risky.
While rotation is a viable option,
  • There is a lot of manual work involved.
  • Connection interruption occurs.
  • The developer experience is deteriorating.
For this reason, it tends to become a mere formality in actual operation. With IAM authentication,Short-lived tokens + IAM permissionsThis allows for secure connection and enables "one-time access."

Considerations (Limitations)

Currently, IAM authentication has numerous limitations that cannot be ignored.

Limitations in ElastiCache

excerpt
  • IAM authentication is only available with ElastiCache for Valkey 7.2 or later, or Redis OSS 7.0 or later.
  • IAM authentication tokens expire after 15 minutes, and connections established with IAM authentication are automatically disconnected after 12 hours.

Limitations in RDS

excerpt
  • IAM database authentication does not support all IAM global condition keys.
  • In PostgreSQL, granting a user (including the Master user) the rds_iam role disables password authentication.
  • 300〜1000 MiB
  • 1 秒あたり200

ElastiCache IAM Authentication

I primarily use Redis in Replication Groups. Some behavior differs with Serverless and Valkey.

ElastiCache Cluster Requirements

  • Valkey or Redis must be a specific version or higher; Memcached is not supported.
  • TLS required
    • You cannot specify a UserGroup unless TLS is required.
    • NodeType should be fine as long as it's the latest generation.

ElastiCache User Group Requirements

  • Redis
    • Make sure to include the Default User.
      • Create a different default user with UserName=default and UserID.
      • The actual IAM authentication user is created separately.
    • The default user rejects all access strings and disables password authentication.
    • Valkey does not require a default user.
  • For IAM authentication, the UserName and UserID must match.

Important points regarding TLS (mistranslation of official documentation)

The Japanese document contains the following mistranslations:
Encryption during transit is only supported in replication groups running the following node types: M1, M2.
The original English text is below.
In-transit encryption is not supported for replication groups running the following node types: M1, M2.
The meaning is the exact opposite. Newer NodeTypes now allow you to enable TLS.

Known bugs in the Terraform Provider and steps to change TLS settings.

In some cases, enabling TLS and specifying a User Group simultaneously in an existing Replication Group may result in encryption-in-transit not being applied correctly.
I'm using a method where I manually make the changes and then use Terraform to make them match.
  1. Manual change
      • transit_encryption_enabled = true
      • transit_encryption_mode = preferred
  1. Manual change
      • transit_encryption_mode = required
  1. UserGroup Specification
      • Specify "User Group" as the Access Control method and specify "user_group_ids" (Terraform execution is possible).
Furthermore, when enabling TLS on an existing Serverless/Valkey cluster with TLS disabled, the authentication mode cannot be directly changed; therefore, the Preferred -> Required procedure must be followed. Outside of ReplicationGroups, this change can be made not manually, but through multiple executions using Terraform.

ElastiCache IAM Authentication Setup Flow

  1. Create an ElastiCache user for IAM authentication.
  1. Attach an IAM Policy to a Pod/EC2/IAM Role
Specifically, this is the policy. This policy can be restricted by Resource Tag.

Connection Flow

  1. Generate an IAM authentication token with SigV4
    1. Unfortunately, as of December 2025, there is no dedicated CLI for generating UserTokens, so you need to generate SigV4 Tokens programmatically. The official documentation has an example in Java, but I tried writing one in Go.
  1. Connect to Redis using TLS
Example of IAM authentication token generation (Go)
Configuration of the generated tokens
redis-cli connection example

RDS IAM Authentication

IAM authentication for RDS is significantly simpler than for ElastiCache. There are mainly two requirements.
  • IAM authentication enablement settings within the cluster
  • Specifying user roles and AWSAuthenticationPlugin using SQL statements
iam_database_authentication_enabled = true
I also recommend specifying the log; it's easier to use when troubleshooting.
enabled_cloudwatch_logs_exports = ["iam-db-auth-error"]
 
Regarding SQL, the method for specifying the master user differs between MySQL and PostgreSQL.
Since I have only tested PostgreSQL, the information may be biased.
Generally, it's best to target the Master user with IAM authentication, but since password authentication will become unavailable the moment you switch to IAM authentication, you should consider how to handle the switch on the application side.

About IAM Policies

I believe IAM policies are the biggest weakness of RDS IAM authentication.
connect
connect
Furthermore, it does not support any Condition Keys, including Resource Tags. Only ARNs are supported.
Cluster ID
This is what it would look like if written in Terraform.

Connection Flow

  1. Generate an IAM authentication token (Token generation can be done with the AWS CLI).
  1. Connect to the database using TLS.
 
The Token is specified as the Client's password.
—sslmode=verify-full sslrootcert=full_path_to_ssl_certificate

Other points that make you want to play

aws:SourceIp
The ElastiCache documentation states the following:
When using IAM authentication with replication groups, aws:SourceIp and aws:ResourceTag/%s are supported.
However, this turned out to be a lie.
aws:SourceIp

In conclusion

IAM authentication allows for infrastructure configurations that don't rely on password management, but if implemented without understanding the limitations regarding token handling and conditional keys, you're guaranteed to run into problems.
Furthermore, the official documentation for IAM authentication is insufficient in several areas. It lacks sufficient explanation for implementation and operation scenarios, requiring additional verification to correctly understand the specifications. I, too, proceeded with implementation based on the official documentation, but incurred unnecessary verification costs before arriving at the correct configuration.
Compared to password authentication, IAM authentication has a higher barrier to entry and places greater implementation responsibilities on the application side. Therefore, we believe that IAM authentication should not be treated simply as a convenient feature, but rather as a mechanism that should be chosen only after understanding its use cases and prerequisites. We hope this article will provide some useful information for understanding IAM authentication in this way.
SRG is looking for new team members.
If you are interested, please contact us here.