Skip to content

How can I add an existing s3 bucket as a lambda trigger using AWS Cloudformation template? Is it even possible to do so using only CloudFormation template?

0

I want to add a existing S3 bucket as a trigger for a lambda function that i am creating in the stack. I do not want to use management console to import the s3 bucket resource as i have to repeat this process multiple times with different s3 buckets.

Here is a sample of code that i am using for adding a s3 trigger for lambda whenever a txt file is uploaded in the bucket :

AWSTemplateFormatVersion: 2010-09-09
Description:  Adding Lambda trigger using existing s3 bucket
Parameters:
  LambdaS3Bucket:
    Type: String
    Description: S3 Lambda bucket
    Default: my-lambda-code-bucket
  TestTriggerBucketName:
    Type: String
    Description: S3 bucket that will work as trigger for the lambda
    Default: my-test-trigger-bucket
  

Resources:
  LambdaFunctionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Action:
              - sts:AssumeRole
            Effect: Allow
            Principal:
              Service:
                - lambda.amazonaws.com
      Policies:
        - PolicyDocument:
            Version: "2012-10-17"
            Statement:
              - Effect: Allow
                Action:
                  - logs:CreateLogGroup
                  - logs:CreateLogStream
                  - logs:PutLogEvents
                Resource: "*"
              - Effect: Allow
                Action:
                  - s3:GetObject
                Resource: "arn:aws:s3:::*/*"
          PolicyName: !Sub "lambda-policy-role"
 
  TestLambdaTrigger:
    Type: AWS::Lambda::Function
    Properties:
      Code:
        S3Bucket: !Ref LambdaS3Bucket
        S3Key: lambda-code.zip # lambda code
      FunctionName: "lambda-function-trigger-test"
      Handler: lambda_function.lambda_handler
      Role: !GetAtt LambdaFunctionRole.Arn
      Runtime: python3.11
      Description: "Testing trigger"

  TestTriggerS3:
    Type: AWS::S3::Bucket
    DependsOn: TestTriggerInvokePermission
    Properties:
      BucketName: !Ref TestBucketName
      NotificationConfiguration:
        LambdaConfigurations:
          - Event: s3:ObjectCreated:Put
            Filter:
              S3Key:
                Rules:
                  - Name: suffix
                    Value: txt
            Function: !GetAtt TestLambdaTrigger.Arn
  
  TestTriggerInvokePermission:
    Type: AWS::Lambda::Permission
    DependsOn: TestLambdaTrigger
    Properties:
      FunctionName: !GetAtt TestLambdaTrigger.Arn
      Action: lambda:InvokeFunction
      Principal: s3.amazonaws.com
      SourceArn: !Sub arn:aws:s3:::${TestTriggerBucketName}

The above code creates the s3 bucket and then adds the trigger. I require to do the same without creating the s3 bucket. Basically i want to remove the creation of this part:

 TestTriggerS3:
    Type: AWS::S3::Bucket
    DependsOn: TestTriggerInvokePermission
    Properties:
      BucketName: !Ref TestBucketName
      NotificationConfiguration:
        LambdaConfigurations:
          - Event: s3:ObjectCreated:Put
            Filter:
              S3Key:
                Rules:
                  - Name: suffix
                    Value: txt
            Function: !GetAtt TestLambdaTrigger.Arn
1 Answer
4

You’re correct that adding an S3 bucket trigger to a Lambda function using CloudFormation is impossible if the bucket already exists and isn’t defined within the same stack, since NotificationConfiguration can only be specified within the AWS::S3::Bucket resource.

This has been part of the CFN coverage roadmap for ages : https://github.com/aws-cloudformation/cloudformation-coverage-roadmap/issues/79

I can think of some workarounds:

  1. SAM : If you're trying to trigger a Lambda function from these S3 events, you can make use of the AWS::Serverless::Function resource and configure its EventSource property to point to S3.

  2. Custom Resource : In the Lambda function for this custom resource, use the AWS SDK to programmatically add an event notification to the existing S3 bucket. The function will call s3.putBucketNotificationConfiguration to set up the event for the Lambda function.

  3. Use EventBridge as a middle layer : This has been described in https://repost.aws/knowledge-center/eventbridge-rule-monitors-s3

Consider that approaches 1 and 2 would result in the bucket-creator stack to appear drifted.

Hope that helps.

AWS
SUPPORT ENGINEER
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.