Lambda concurrency for Self managed Kafka event source


I created an event source mapping between self-managed Kafka and a lambda function following documents this and this.

Lambda function is getting triggered but all batches are getting processed serially. The expectation from the documentation is that there will be concurrent lambdas whose number is equal to the number of partitions in that topic. I am not observing this.

I want to understand under what conditions, threshold or ranges, for consumer_lag and consumer_offset for which one can observe concurrent Lambda invocations and scaling.

asked 2 years ago684 views
1 Answer

I understand by Self-Managed Kafka, that you are not using the AWS MSK service and have installed kafka somewhere in AWS infra. My case is exactly similar except that I am using MSK.

answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions