How can I use a CloudFormation resource import to create an Amazon S3 notification configuration for Lambda on an existing S3 bucket?

Short description

To configure an Amazon S3 notification without using a custom resource, do the following:

1. Create a template with the Lambda function S3NotificationLambdaFunction. This function adds the existing bucket NotificationS3Bucket notification configuration.
2. Use a resource import to bring the existing NotificationS3Bucket S3 bucket that's specified in the template into CloudFormation management.
3. Update the CloudFormation stack to include the properties that you want to activate in your S3 bucket.

To use a custom resource, see How can I use CloudFormation to create an Amazon S3 notification configuration for Lambda on an existing S3 bucket?

Resolution

Important: The following steps override any existing or manually created notification configurations in an S3 bucket. Follow these steps to add a new notification configuration to an imported S3 bucket.

Create a CloudFormation template with a Lambda function

The following example template creates a Lambda function with a running role and permissions to invoke the function. To use an existing Lambda function, use the function's Amazon Resource Name (ARN) in your CloudFormation template for the LambdaConfiguration property in the S3 bucket.

AWSTemplateFormatVersion: 2010-09-09
Description: >-
  Sample template that shows you how to import an existing S3 bucket as an event source for a Lambda function
Parameters:
  NotificationBucket:
    Type: String
    Description: S3 bucket that's used for Lambda event notification
Resources:
  S3NotificationLambdaFunction:
    Type: 'AWS::Lambda::Function'
    Properties:
      Code:
        ZipFile: !Join
          - |+

          - - import json
            - 'def lambda_handler(event,context):'
            - '    return ''Welcome... This is a test Lambda Function'''
      Handler: index.lambda_handler
      Role: !GetAtt LambdaIAMRole.Arn
      Runtime: python3.6
      Timeout: 5
  LambdaInvokePermission:
    Type: 'AWS::Lambda::Permission'
    Properties:
      FunctionName: !GetAtt S3NotificationLambdaFunction.Arn
      Action: 'lambda:InvokeFunction'
      Principal: s3.amazonaws.com
      SourceAccount: !Ref 'AWS::AccountId'
      SourceArn: !Sub 'arn:aws:s3:::${NotificationBucket}'
  LambdaIAMRole:
    Type: 'AWS::IAM::Role'
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - lambda.amazonaws.com
            Action:
              - 'sts:AssumeRole'
      Path: /
      Policies:
        - PolicyName: root
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - 's3:GetBucketNotification'
                  - 's3:PutBucketNotification'
                Resource: !Sub 'arn:aws:s3:::${NotificationBucket}'
              - Effect: Allow
                Action:
                  - 'logs:CreateLogGroup'
                  - 'logs:CreateLogStream'
                  - 'logs:PutLogEvents'
                Resource: 'arn:aws:logs:*:*:*'

Import an existing S3 bucket to your CloudFormation stack

1.    Open the AWS CloudFormation console.

2.    On the navigation pane, choose Stack, and then select the stack that you created.

3.    Choose Stack actions, and then choose Import resources into stack.

4.    Review the Import overview page, and then choose Next.

Important: In your CloudFormation template, each resource that you import must have a DeletionPolicy attribute, and all other resources must remain the same. For example:

AWSTemplateFormatVersion: 2010-09-09
Description: >-
  Sample template that shows you how to import an existing S3 bucket as an event source for a Lambda function
Parameters:
  NotificationBucket:
    Type: String
    Description: S3 bucket that's used for Lambda event notification
Resources:
  S3NotificationLambdaFunction:
    Type: 'AWS::Lambda::Function'
    Properties:
      .
      .
  LambdaInvokePermission:
    Type: 'AWS::Lambda::Permission'
    Properties:
      .
      .
  LambdaIAMRole:
    Type: 'AWS::IAM::Role'
    Properties:
      .
      .

  <The preceding resources were already created. Now, add the DeletionPolicy to the preceding stack to import the S3 bucket.>
  NotificationS3Bucket:
    Type: 'AWS::S3::Bucket'
    DeletionPolicy: Retain
    Properties:
      BucketName: myenv-bucket          #Bucket name to import

5.    In the Specify template section, choose Amazon S3 URL or Upload a template file based on your requirements, and then choose Next.

6.    Complete the rest of the steps in the wizard to import your existing resources. For more information, see Import an existing resource into a stack using the AWS CloudFormation console.

7.    Wait for the stack to reach the IMPORT_COMPLETE state.

Update your CloudFormation template

1.    Update the stack with the CloudFormation template that you modified.

2.    Wait for the stack to reach the UPDATE_COMPLETE state, and then verify the NotificationConfiguration on the S3 bucket.

3.    Turn on Amazon S3 event notifications on your S3 bucket. In the following example template, the bucket name is myenv-bucket:

AWSTemplateFormatVersion: 2010-09-09
Description: >-
  Sample template that shows you how to import an existing S3 bucket as an event source for a Lambda function
Parameters:
  NotificationBucket:
    Type: String
    Description: S3 bucket that's used for Lambda event notification
Resources:
  S3NotificationLambdaFunction:
    Type: 'AWS::Lambda::Function'
    Properties:
      .
      .
  LambdaInvokePermission:
    Type: 'AWS::Lambda::Permission'
    Properties:
      .
      .
  LambdaIAMRole:
    Type: 'AWS::IAM::Role'
    Properties:
      .
      .

  <The preceding resources were already created. Now, add the DeletionPolicy to the preceding stack to import the S3 bucket.>
  NotificationS3Bucket:
    Type: 'AWS::S3::Bucket'
    DeletionPolicy: Retain
    Properties:
      BucketName: myenv-bucket
      NotificationConfiguration:   #Update stack with NotificationConfiguration
          LambdaConfigurations:
              - Event: 's3:ObjectCreated:Put'
                Function: !GetAtt S3NotificationLambdaFunction.Arn
              - Event: 's3:ObjectRemoved:*'
                Function: !GetAtt S3NotificationLambdaFunction.Arn