Using MSK as trigger to a Lambda with SASL/SCRAM Authentication


Hi, I have set up a MSK cluster with SASL/SCRAM authentication. I have stored the username and password in a secret using AWS Secrets Manager. Now I am trying to set the topic in the MSK cluster as an event source to a Lambda function. In order to do so, I am following this documentation: However the above documentation is for unauthenticated protocol. So I tried to add the authentication and the secret. I also added a policy in the execution role of the Lambda function that lets it read the secret value:

"Version": "2012-10-17",
"Statement": [
        "Effect": "Allow",
        "Action": [
        "Resource": [
        "Effect": "Allow",
        "Action": "secretsmanager:ListSecrets",
        "Resource": "*"

When I am trying to add the trigger, I am getting the error: An error occurred when creating the trigger:

Cannot access secret manager value arn:aws:secretsmanager:::secret:*******. Please ensure the role can perform the 'secretsmanager:GetSecretValue' action on your broker in IAM. (Service: AWSLambda; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: ****; Proxy: null)

I am not able to understand this error since I have included in the policy all the Actions from "secretsmanager" on all the Resources in my account. Can someone help?

2 Answers

Hi, Thanks for your reply. Since I would like the MSK Cluster have public access, I cannot have clusters hosted in private subnets inside the VPC. I do have a NAT Gateway running in the Public subnet. I am able to produce and consume messages to the topic in this cluster from terminal, but when trying to redirect the message to the Lambda function as a payload, I am facing the above mentioned problem.

answered 2 years ago
  • Are you deploying the Lambda function with VPC access? Is that the same VPC where the MSK cluster is running?


Since Lambda must have access to your VPC to use Amazon MSK as a trigger, you need to make sure that, if the subnets you chose are private, there is connectivity to a NAT Gateway or to a Secrets Manager VPC Endpoint to access Secrets Manager.

profile pictureAWS
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions