DynamoDB Stream Event Source for Java & Python lambda behaves differently with partial failures

0

Hi everyone,

I created 2 lamba (1 in Python & 1 in Java) from code https://docs.aws.amazon.com/lambda/latest/dg/with-ddb.html#services-ddb-batchfailurereporting The only difference is that I returned Partial failure for every records. Both lambdas were configured to be triggered by DynamoDB Stream with partial failure enable & 2 retry attempts (same configuration for both). However, while python code runs messages three times (first time & 2 retries) - Java lambda runs once (without any retry).

Is this a bug in AWS code or did I miss something?

demandé il y a 6 mois312 vues
1 réponse
1

Without seeing your code it's impossible to tell. But my assumption is that you are using ReportBatchItemFailures without actually returning the failures, in which case no retries are happening as Lambda thinks everything was successful. For example:

try:
    # some Lambda logic
except Exception as e:
    return { "batchItemFailures": [ {"itemIdentifier": DYNAMODB RECORD SEQUENCE NUMBER} ]  }
profile pictureAWS
EXPERT
répondu il y a 6 mois
  • Hi thanks for being interested in my question. Here are my code: Python:

    import json
    
    def handler(event, context):
        records = event.get("Records")
        curRecordSequenceNumber = "";
        
        for record in records:
            curRecordSequenceNumber = record["dynamodb"]["SequenceNumber"]
            print({"batchItemFailures":[{"itemIdentifier": curRecordSequenceNumber}]})
            return {"batchItemFailures":[{"itemIdentifier": curRecordSequenceNumber}]}
    
        return {"batchItemFailures":[]}
    

    Java:

    ... // Some import
    
    public class TestStreamHandler implements RequestHandler<DynamodbEvent, Serializable> {
        LambdaLogger logger;
        final static ObjectMapper objectMapper = new ObjectMapper();
    
        @Override
        public StreamsEventResponse handleRequest(DynamodbEvent input, Context context) {
            logger = context.getLogger();
            List<StreamsEventResponse.BatchItemFailure> batchItemFailures = new ArrayList<>();
            String curRecordSequenceNumber = "";
    
            for (DynamodbEvent.DynamodbStreamRecord dynamodbStreamRecord : input.getRecords()) {
                StreamRecord dynamodbRecord = dynamodbStreamRecord.getDynamodb();
                curRecordSequenceNumber = dynamodbRecord.getSequenceNumber();
    
                batchItemFailures.add(new StreamsEventResponse.BatchItemFailure(curRecordSequenceNumber));
                return new StreamsEventResponse(batchItemFailures);
            }
            return new StreamsEventResponse();
    }
    
  • Here is my event configuration:

    DynamoDB: TestStreamTable
    arn:aws:dynamodb:ap-southeast-1:<myaccount>:table/TestStreamTable/stream/2023-11-18T15:42:39.936
    state: Enabled
    Details
    Activate trigger: Yes
    Batch size: 100
    Batch window: None
    Concurrent batches per shard: 1
    Last processing result: OK
    Maximum age of record: -1
    On-failure destination: arn:aws:sqs:ap-southeast-1:<myaccount>:TestStreamFuncPythonSQS
    Report batch item failures: Yes
    Retry attempts: 2
    Split batch on error: Yes
    Starting position: TRIM_HORIZON
    Tumbling window duration: None
    UUID: <id>
    

    And for Java:

    DynamoDB: TestStreamTable
    arn:aws:dynamodb:ap-southeast-1:<myaccount>:table/TestStreamTable/stream/2023-11-18T15:42:39.936
    state: Enabled
    Details
    Activate trigger: Yes
    Batch size: 100
    Batch window: None
    Concurrent batches per shard: 1
    Last processing result: OK
    Maximum age of record: -1
    On-failure destination: arn:aws:sqs:ap-southeast-1:<myaccount>:TestStreamFuncJavaSQS
    Report batch item failures: Yes
    Retry attempts: 2
    Split batch on error: Yes
    Starting position: TRIM_HORIZON
    Tumbling window duration: None
    UUID: <id>
    

    Could you help to have a look?

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions