I want to make my AWS Lambda function code idempotent to prevent inconsistencies and data loss in my application.
In programming, idempotency refers to the capacity of an application or component to identify repeated events and prevent duplicated, inconsistent, or lost data. Making your AWS Lambda function idempotent requires designing your function logic to treat duplicated events correctly.
Idempotent function logic can help reduce the following issues:
- Unnecessary API calls
- Code processing time
- Data inconsistency
To make your function idempotent, the function's code must properly validate input events and identify if the events were processed before. Your application's functionality dictates the best way to write the code.
Use the following example function logic and best practices for guidance.
Idempotent Lambda function logic example
Note: The following example Lambda function logic applies to most use cases.
1. Extract the value of a unique attribute of the input event. (For example, a transaction or purchase ID.)
2. Check if the attribute value exists in a control database. (For example, an Amazon DynamoDB table.)
3. Depending on the results, complete the following step:
If a unique value exists, then end the action without producing an error.
If a unique value doesn't exist, then proceed with the action that you originally designed.
Note: Adding AWS services to your architecture might incur additional costs. For more information, see Amazon DynamoDB pricing and AWS Pricing.
4. If the unique value doesn’t exist after the function work finishes, then include the record in the control database.
5. Finish the action.
Note: If you use Java or Python runtimes for your function, then review the Python and Java idempotency modules on the AWS Lambda Powertools GitHub page.
Lambda function idempotency best practices
- Plan your idempotency features before developing your application.
- When your Lambda code processes a duplicate event, make sure that it ends without producing an error.
Note: Throwing errors can cause further retries, either by Lambda or by another service that invokes your function.
- Change the Lambda function timeout setting, so that the full runtime is handled correctly.
Note: Using a separate service to persist data and control duplicated events might require API calls to HTTPS endpoints. API calls to HTTPS endpoints might then require more runtime than the default 3 seconds.
- Test and optimize your function as much as possible. Simulate a real scenario and rate of requests.
Note: It's critical to test and optimize idempotent function logic to help prevent potential side effects such as, timeouts, excessive latency, or bottlenecks.
- Store session data using a service that's easily scalable and provides high throughput, such as DynamoDB.
Note: When making API calls to Amazon Elastic Compute Cloud (Amazon EC2), the service offers the parameter clientToken. This parameter makes sure that a mutating API request successfully completes its workflow only once, even if you initiate multiple retries with the same clientToken.
Lambda programming model
Create a Lambda function with the console
Making retries safe with idempotent APIs