DynamoDB Stream Event Source for Java & Python lambda behaves differently with partial failures

Hi everyone,

I created 2 lamba (1 in Python & 1 in Java) from code https://docs.aws.amazon.com/lambda/latest/dg/with-ddb.html#services-ddb-batchfailurereporting The only difference is that I returned Partial failure for every records. Both lambdas were configured to be triggered by DynamoDB Stream with partial failure enable & 2 retry attempts (same configuration for both). However, while python code runs messages three times (first time & 2 retries) - Java lambda runs once (without any retry).

Is this a bug in AWS code or did I miss something?

トピック

サーバーレス計算するデータベース

タグ

AWS Lambda Amazon DynamoDB

言語

English

Trung Anh Nuoa

質問済み 6ヶ月前313ビュー

1回答

新しい順
投票が多い順
コメントが多い順

Without seeing your code it's impossible to tell. But my assumption is that you are using ReportBatchItemFailures without actually returning the failures, in which case no retries are happening as Lambda thinks everything was successful. For example:

try:
    # some Lambda logic
except Exception as e:
    return { "batchItemFailures": [ {"itemIdentifier": DYNAMODB RECORD SEQUENCE NUMBER} ]  }

エキスパート

Leeroy Hannigan

回答済み 6ヶ月前

Trung Anh Nuoa

6ヶ月前

Hi thanks for being interested in my question. Here are my code: Python:

import json

def handler(event, context):
    records = event.get("Records")
    curRecordSequenceNumber = "";
    
    for record in records:
        curRecordSequenceNumber = record["dynamodb"]["SequenceNumber"]
        print({"batchItemFailures":[{"itemIdentifier": curRecordSequenceNumber}]})
        return {"batchItemFailures":[{"itemIdentifier": curRecordSequenceNumber}]}

    return {"batchItemFailures":[]}

Java:

... // Some import

public class TestStreamHandler implements RequestHandler<DynamodbEvent, Serializable> {
    LambdaLogger logger;
    final static ObjectMapper objectMapper = new ObjectMapper();

    @Override
    public StreamsEventResponse handleRequest(DynamodbEvent input, Context context) {
        logger = context.getLogger();
        List<StreamsEventResponse.BatchItemFailure> batchItemFailures = new ArrayList<>();
        String curRecordSequenceNumber = "";

        for (DynamodbEvent.DynamodbStreamRecord dynamodbStreamRecord : input.getRecords()) {
            StreamRecord dynamodbRecord = dynamodbStreamRecord.getDynamodb();
            curRecordSequenceNumber = dynamodbRecord.getSequenceNumber();

            batchItemFailures.add(new StreamsEventResponse.BatchItemFailure(curRecordSequenceNumber));
            return new StreamsEventResponse(batchItemFailures);
        }
        return new StreamsEventResponse();
}

Trung Anh Nuoa

6ヶ月前

Here is my event configuration:

DynamoDB: TestStreamTable
arn:aws:dynamodb:ap-southeast-1:<myaccount>:table/TestStreamTable/stream/2023-11-18T15:42:39.936
state: Enabled
Details
Activate trigger: Yes
Batch size: 100
Batch window: None
Concurrent batches per shard: 1
Last processing result: OK
Maximum age of record: -1
On-failure destination: arn:aws:sqs:ap-southeast-1:<myaccount>:TestStreamFuncPythonSQS
Report batch item failures: Yes
Retry attempts: 2
Split batch on error: Yes
Starting position: TRIM_HORIZON
Tumbling window duration: None
UUID: <id>

And for Java:

DynamoDB: TestStreamTable
arn:aws:dynamodb:ap-southeast-1:<myaccount>:table/TestStreamTable/stream/2023-11-18T15:42:39.936
state: Enabled
Details
Activate trigger: Yes
Batch size: 100
Batch window: None
Concurrent batches per shard: 1
Last processing result: OK
Maximum age of record: -1
On-failure destination: arn:aws:sqs:ap-southeast-1:<myaccount>:TestStreamFuncJavaSQS
Report batch item failures: Yes
Retry attempts: 2
Split batch on error: Yes
Starting position: TRIM_HORIZON
Tumbling window duration: None
UUID: <id>

Could you help to have a look?

DynamoDB Stream Event Source for Java & Python lambda behaves differently with partial failures

関連するコンテンツ