Cross account event source mapping of kinesis through cloud formation

0

Error Message - Resource handler returned message: "Invalid request provided: Cannot access stream arn, Please ensure the role can perform the GetRecords, GetShardIterator, DescribeStream, DescribeStreamSummary, ListShards, and ListStreams Actions on your stream. (Service: Lambda, Status Code: 400, Request ID: )" (RequestToken: , HandlerErrorCode: InvalidRequest)

Setup

Added resource based policy to kinesis streams with the lambda exec role as principal and added the following actions "kinesis:DescribeStreamSummary", "kinesis:ListShards", "kinesis:GetRecords", "kinesis:GetShardIterator"

Added inline policy for lambda to access the stream with the above actions and kinesis stream arn as principal.

2 Answers
0

The error suggests that the Lambda function doesn't have sufficient permissions to access the Kinesis stream. To resolve:

  1. First, ensure the Lambda execution role has ALL the required permissions. You're missing few required permissions in your setup:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "kinesis:DescribeStream",           // This one was missing
                "kinesis:DescribeStreamSummary",
                "kinesis:GetRecords",
                "kinesis:GetShardIterator",
                "kinesis:ListShards",
                "kinesis:ListStreams"     // This one was missing
            ],
            "Resource": "arn:aws:kinesis:REGION:ACCOUNT-ID:stream/STREAM-NAME"
        }
    ]
}
  1. Verify your resource-based policy on the Kinesis stream is correct. Here's an example:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::ACCOUNT-ID:role/YOUR-LAMBDA-ROLE-NAME"
            },
            "Action": [
                "kinesis:DescribeStream",
                "kinesis:DescribeStreamSummary",
                "kinesis:GetRecords",
                "kinesis:GetShardIterator",
                "kinesis:ListShards",
                "kinesis:ListStreams"
            ],
            "Resource": "arn:aws:kinesis:REGION:ACCOUNT-ID:stream/STREAM-NAME"
        }
    ]
}

If the issue persists after implementing these changes:

  1. Try testing the permissions using the AWS Policy Simulator - [https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_testing-policies.html]
  2. Check if there are any organization-level SCPs (Service Control Policies) that might be restricting access
AWS
answered a month ago
-1

The error message you're encountering suggests that there's an issue with the permissions setup for your cross-account event source mapping between Kinesis and Lambda. While you've taken some good steps, there are a few things to check and potentially adjust:

  1. Ensure that you've included all necessary permissions in both the resource-based policy on the Kinesis stream and the inline policy for the Lambda function. The error message mentions "DescribeStream" and "ListStreams" actions, which are not in your list. Add these to both policies:

    "kinesis:DescribeStream", "kinesis:ListStreams"

  2. Double-check that you've applied the resource-based policy to the correct Kinesis stream in the source account.

  3. Verify that the Lambda execution role ARN in the resource-based policy on the Kinesis stream is correct and matches the role associated with your Lambda function.

  4. In the CloudFormation template for your Lambda function, make sure you're using the correct EventSourceArn for the Kinesis stream. This should be the full ARN of the stream in the other account.

  5. Ensure that the StartingPosition for the event source mapping is set correctly (e.g., "TRIM_HORIZON" or "LATEST").

  6. If you're using enhanced fan-out, you may need to add additional permissions for the consumer, such as:

    "kinesis:DescribeStreamConsumer", "kinesis:SubscribeToShard"

  7. Check that your Lambda function's execution role has the necessary permissions to interact with CloudWatch Logs for function logging.

If you've verified all these points and are still encountering issues, you may need to review the full IAM setup, ensuring that there are no conflicting policies or restrictions preventing the cross-account access. Remember that when working with cross-account resources, permissions need to be set up correctly on both sides of the interaction.
Sources
Process Amazon Kinesis Data Streams records with Lambda - AWS Lambda
AWS::Lambda::EventSourceMapping - AWS CloudFormation
Invoke AWS Lambda functions from cross-account Amazon Kinesis Data Streams | AWS Big Data Blog

profile picture
answered a month ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions