Skip to content

Secrets Rotation with Zero-Delay Refresh Across Multiple Services

0

I have n numbers of Java backend services running that currently load secrets (DB creds, API keys, etc.) from AWS Secrets Manager at startup and then cache them in memory.

The challenge:

When AWS Secrets Manager rotates a secret, my services keep using the stale/expired secret until restart or manual refresh.

I want a solution where services immediately refresh secrets with zero delay (even <1 sec) when rotation happens.

Is there an AWS-native/in-built mechanism within secrets code to automatically fetch when secrets rotated most of the system won't auto fetch until if someone manually triggers to fetch new secrets but AWS get secrets method should auto fetch the new secrets and inform code to reinstate bean.

2 Answers
6

How about by leveraging WS-Native Strategy for Instant Secret Refresh:

  1. Use AWS Secrets Manager Caching Library (Java) • AWS provides a Java caching client that supports configurable refresh intervals. • You can set the refresh interval to a very low value (e.g., 1 second), but this still relies on polling, not event-driven updates.
SecretCacheConfiguration config = new SecretCacheConfiguration()
    .withCacheItemTTL(1000); // 1 second TTL
SecretCache cache = new SecretCache(config);
String secret = cache.getSecretString("your-secret-id");

This increases API calls and cost. Use with caution and consider batching or throttling.

  1. Event-Driven Refresh via AWS Lambda + SNS/SQS • Secrets Manager rotation can trigger a Lambda function. • That Lambda can publish a message to SNS or push to SQS. • Your Java services subscribe to SNS or poll SQS and refresh secrets when notified.

  2. Spring Boot + AWS SecretsManager JDBC Driver • For DB credentials, use the AWS SecretsManager JDBC wrapper which auto-refreshes secrets on authentication failure. • It retries with fresh credentials if login fails, avoiding manual refresh.

spring:
  datasource:
    url: jdbc-secretsmanager:mysql://your-db-host:3306/dbname
    username: secret/db-creds
    driver-class-name: com.amazonaws.secretsmanager.sql.AWSSecretsManagerMySQLDriver
  1. Custom Refresh Bean with Scheduled Polling • For non-DB secrets (e.g., API keys), implement a Spring @Scheduled task that checks for secret changes every few seconds. • Compare current secret hash with the latest from Secrets Manager. • If changed, refresh the bean or reload the config.
EXPERT

answered a year ago

0

Hi, There is no one solution that fits your criteria. When caching the secrets on startup, that compute is designed not to fetch secrets until a restart/ refresh. The alternate of fetching secrets every time you need them may be too intensive but may work for some.

A few options to consider:

  • For database secrets
    • Using JDBC connections, a secrets manager wrapper is available to manage the connections directly. More info here
    • When using ORMs and other connection pools, custom code is required to fetch the secrets and refresh the pool, that matches with the rotation window. Leverage two different accounts as blue/ green secret where both accounts and passwords exist for a few weeks to update applications accordingly.
  • For other secrets
    • Consider retrieving from secrets manager using Java caching to fetch the secrets and configure the refresh interval or retry to fetch on error.

While rotation can be automated in many ways and some services manage rotation natively, fetching the secrets immediately once rotated is a trade off decision left to the individual workloads vs. as a native mechanism.

answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.