(SAM CLI 1.137.1) circular dependency reported by SAM Validate when s3 event trigger added to lambda

0

In my template.yaml, when I connect a lambda to an s3 bucket in the Infrastructure Composer (running in VS Code) it creates an Events: property in the lambda.

  handleS3uploadFunction:
    Type: AWS::Serverless::Function
    Properties:
      Description: !Sub
        - Stack ${AWS::StackName} Function ${ResourceName}
        - ResourceName: handleS3uploadFunction
      CodeUri: backend/
      Handler: src/handlers/handle-s3-upload.s3UploadHandler
      Runtime: nodejs22.x
      MemorySize: 3008
      Timeout: 20
      Tracing: Active
      Environment: {}
      Policies:
        - DynamoDBCrudPolicy:
            TableName: !Ref photoProvTable
      # TODO: hooking up event trigger causes a circular dependency. :(  Not sure how to resolve
      # Events:
      #   s3Trigger:
      #     Type: S3
      #     Properties:
      #       Bucket: !Ref photosS3Bucket
      #       Events:
      #         - s3:ObjectCreated:*
      #         - s3:ObjectRemoved:*

But when I run the SAM validate command, it states there is a circular reference formed.

[[E3004: Resource dependencies are not circular] (Circular Dependencies for resource photosS3Bucket. Circular dependency with [handleS3uploadFunctions3TriggerPermission]) matched 301, [E3004: Resource dependencies are not circular] (Circular Dependencies for resource handleS3uploadFunctions3TriggerPermission. Circular dependency with [handleS3uploadFunction]) matched 14, [E3004: Resource dependencies are not circular] (Circular Dependencies for resource handleS3uploadFunction. Circular dependency with [photosS3Bucket]) matched 512]
Error: Linting failed. At least one linting rule was matched to the provided template.

How do I resolve this? I tried following the solution in several postings I found on Stack Overflow, but the circular dependency error didn't go away.

I was using SAM version 1.135 when this happened. Upgraded to 1.137 and it's still a problem.

Thanks for your help! -Matt


"Full" template (some unrelated resources removed due to post text limit)

AWSTemplateFormatVersion: 2010-09-09
Description: stack desc
Transform: AWS::Serverless-2016-10-31

Parameters:
  SiteName:
    Type: String
    Description: The name of the site (used for resource naming).
    Default: itsasite
    # AllowedPattern: "[a-zA-Z0-9-]+" # Optional: Restrict the allowed characters
    # MinLength: 3 # Optional: Minimum length
    # MaxLength: 63 # Optional: Maximum length

Resources:

  AdminWebAppBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Sub ${SiteName}-admin-web-app-dist
      BucketEncryption:
        ServerSideEncryptionConfiguration:
          - BucketKeyEnabled: true
      VersioningConfiguration:
        Status: Enabled
  AdminWebAppBucketPolicy:
    Type: AWS::S3::BucketPolicy
    Properties:
      Bucket: !Ref AdminWebAppBucket
      PolicyDocument:
        Version: "2012-10-17"
        Id: PolicyForCloudFrontPrivateContent
        Statement:
          - Sid: AllowCloudFrontServicePrincipal
            Effect: Allow
            Principal:
              Service: cloudfront.amazonaws.com
            Action: s3:GetObject
            Resource: !Join
              - ""
              - - "arn:aws:s3:::"
                - !Ref AdminWebAppBucket
                - /*
            Condition:
              StringEquals:
                # orig... AWS:SourceArn: !Join ['', ['arn:aws:cloudfront::', !Ref "AWS::AccountId", :distribution/, !Ref CloudFrontDistribution]]
                AWS:SourceArn: !Join
                  - ""
                  - - "arn:aws:cloudfront::"
                    - !Ref AWS::AccountId
                    - ":distribution/"
                    - !Ref AdminWebAppCFDistribution
  CloudFrontOriginAccessControl:
    Type: AWS::CloudFront::OriginAccessControl
    Properties:
      OriginAccessControlConfig:
        Name: !Sub ${AdminWebAppBucket} OAC
        OriginAccessControlOriginType: s3
        SigningBehavior: always
        SigningProtocol: sigv4
  ApplicationResourceGroup:
    Type: AWS::ResourceGroups::Group
    Properties:
      Name: !Sub ApplicationInsights-SAM-${AWS::StackName}
      ResourceQuery:
        Type: CLOUDFORMATION_STACK_1_0
  ApplicationInsightsMonitoring:
    Type: AWS::ApplicationInsights::Application
    Properties:
      ResourceGroupName: !Ref ApplicationResourceGroup
      AutoConfigurationEnabled: "true"
  laMagicAdminApi:
    Type: AWS::Serverless::Api
    Properties:
      Name: !Sub ${SiteName}-admin-api
      StageName: Prod
      DefinitionBody:
        openapi: "3.0"
        info: {}
        paths:
          /shows:
            get:
              x-amazon-apigateway-integration:
                httpMethod: POST
                type: aws_proxy
                uri: !Sub arn:${AWS::Partition}:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${adminGetAllShowsFunction.Arn}/invocations
              responses: {}
            put:
              x-amazon-apigateway-integration:
                httpMethod: POST
                type: aws_proxy
                uri: !Sub arn:${AWS::Partition}:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${adminUpsertShowFunction.Arn}/invocations
              responses: {}
          /showdata:
            get:
              x-amazon-apigateway-integration:
                httpMethod: POST
                type: aws_proxy
                uri: !Sub arn:${AWS::Partition}:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${adminGetShowDataFunction.Arn}/invocations
              responses: {}
      EndpointConfiguration: REGIONAL
      TracingEnabled: true
      Cors:
        MaxAge: 5
      Auth:
        Authorizers:
          Cognito:
            UserPoolArn: !GetAtt AdminUserPool.Arn
  AdminUserPool:
    Type: AWS::Cognito::UserPool
    Properties:
      AdminCreateUserConfig:
        AllowAdminCreateUserOnly: true
      AliasAttributes:
        - email
        - preferred_username
      UserPoolName: !Sub ${AWS::StackName}-AdminUserPool
      AutoVerifiedAttributes:
        - email
  preAuthGroupCheck:
    Type: AWS::Serverless::Function
    Properties:
      Description: !Sub
        - Stack ${AWS::StackName} Function ${ResourceName}
        - ResourceName: preAuthGroupCheck
      CodeUri: backend/
      Handler: src/handlers/pre-auth-group-check.checkUserInGroup
      Runtime: nodejs22.x
      MemorySize: 3008
      Timeout: 30
      Tracing: Active
      Environment: {}
  preAuthGroupCheckLogGroup:
    Type: AWS::Logs::LogGroup
    DeletionPolicy: Retain
    UpdateReplacePolicy: Retain
    Properties:
      LogGroupName: !Sub /aws/lambda/${preAuthGroupCheck}
  laMagicTable:
    Type: AWS::DynamoDB::Table
    Properties:
      AttributeDefinitions:
        - AttributeName: id
          AttributeType: S
      BillingMode: PAY_PER_REQUEST
      KeySchema:
        - AttributeName: id
          KeyType: HASH
      StreamSpecification:
        StreamViewType: NEW_AND_OLD_IMAGES

  photosS3Bucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketEncryption:
        ServerSideEncryptionConfiguration:
          - BucketKeyEnabled: true
      VersioningConfiguration:
        Status: Enabled
      BucketName: !Sub ${SiteName}-photos
      CorsConfiguration:
        CorsRules:
          - AllowedHeaders:
              - "*"
            AllowedMethods:
              - GET
              - PUT
              - POST
            AllowedOrigins:
              # TODO: these domains can't be hard-coded for Prod
              - http://localhost:8080
      # NotificationConfiguration:
      #   LambdaConfigurations:
      #     - Event: s3:ObjectCreated:* # Trigger on any object creation event
      #       Function: !GetAtt handleS3uploadFunction.Arn # ARN of the Lambda to trigger

  # CloudFront Distribution for hosting the single page app website
  AdminWebAppCFDistribution:
    Type: AWS::CloudFront::Distribution
    Properties:
      DistributionConfig:
        Origins:
          - DomainName: !GetAtt AdminWebAppBucket.RegionalDomainName
            Id: adminS3Origin
            OriginAccessControlId: !GetAtt CloudFrontOriginAccessControl.Id
            S3OriginConfig:
              OriginAccessIdentity: ""
        Enabled: true
        DefaultRootObject: index.html
        HttpVersion: http2
        DefaultCacheBehavior:
          AllowedMethods:
            - DELETE
            - GET
            - HEAD
            - OPTIONS
            - PATCH
            - POST
            - PUT
          CachedMethods:
            - GET
            - HEAD
          TargetOriginId: adminS3Origin
          ForwardedValues:
            QueryString: false
            Cookies:
              Forward: none
          ViewerProtocolPolicy: allow-all
          MinTTL: 0
          DefaultTTL: 3600
          MaxTTL: 86400
        PriceClass: PriceClass_200
        Restrictions:
          GeoRestriction:
            RestrictionType: whitelist
            Locations:
              - US
              - CA
              - GB
              - DE
        ViewerCertificate:
          CloudFrontDefaultCertificate: true
  
  handleS3uploadFunction:
    Type: AWS::Serverless::Function
    Properties:
      Description: !Sub
        - Stack ${AWS::StackName} Function ${ResourceName}
        - ResourceName: handleS3uploadFunction
      CodeUri: backend/
      Handler: src/handlers/handle-s3-upload.s3UploadHandler
      Runtime: nodejs22.x
      MemorySize: 3008
      Timeout: 20
      Tracing: Active
      Environment: {}
      Policies:
        - DynamoDBCrudPolicy:
            TableName: !Ref laMagicTable
        - Statement:
            - Effect: Allow
              Action:
                - s3:GetObject
                - s3:GetObjectAcl
                - s3:GetObjectLegalHold
                - s3:GetObjectRetention
                - s3:GetObjectTorrent
                - s3:GetObjectVersion
                - s3:GetObjectVersionAcl
                - s3:GetObjectVersionForReplication
                - s3:GetObjectVersionTorrent
                - s3:ListBucket # Needed for some operations, like listing objects if required
                - s3:PutObject # If the function needs to write back to the bucket
                - s3:DeleteObject # If the function needs to delete objects
                # Add other S3 actions as needed by the function's logic
              Resource:
                # Grant access to the bucket itself (for ListBucket)
                - !Sub arn:${AWS::Partition}:s3:::${photosS3Bucket}
                # Grant access to objects within the bucket
                - !Sub arn:${AWS::Partition}:s3:::${photosS3Bucket}/*
      # TODO: hooking up event trigger causes a circular dependency. :(  Not sure how to resolve
      # Events:
      #   s3Trigger:
      #     Type: S3
      #     Properties:
      #       Bucket: photosS3Bucket
      #       Events:
      #         - s3:ObjectCreated:*
      #         - s3:ObjectRemoved:*

  handleS3uploadFunctionLogGroup:
    Type: AWS::Logs::LogGroup
    DeletionPolicy: Retain
    UpdateReplacePolicy: Retain
    Properties:
      LogGroupName: !Sub /aws/lambda/${handleS3uploadFunction}

  # separate permissions statement was added to avoid circular dependencies between handleS3uploadFunction and photos3Bucket (didn't work)
  handleS3uploadFunctionS3Permission:
    Type: AWS::Lambda::Permission
    Properties:
      Action: lambda:InvokeFunction
      FunctionName: !GetAtt handleS3uploadFunction.Arn
      Principal: s3.amazonaws.com
      SourceAccount: !Ref AWS::AccountId
      SourceArn: !GetAtt photosS3Bucket.Arn

Outputs:
  laMagicAdminApiEndpoint:
    Description: API Gateway endpoint URL for Prod stage
    Value: !Sub https://${laMagicAdminApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/
  AdminWebAppBucketName:
    Description: S3 Bucket for deploying Admin web app
    Value: !Ref AdminWebAppBucket
  photosS3Bucket:
    Description: S3 Bucket for storing laMagic photos
    Value: !Ref photosS3Bucket
  laMagicTable:
    Description: DynamoDB table to store laMagic show AWS::DynamoDB::Table
    Value: !Ref laMagicTable

# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
  Function:
    LoggingConfig:
      LogFormat: JSON
    Environment:
      Variables:
        # set resource names here to be used by all lambdas instead of repeating for each lambda
        laMagic_DYNAMODB_TABLE: !Ref laMagicTable
        laMagic_PHOTOS_S3_BUCKET: !Ref photosS3Bucket

  • Can you attach the rest of your template as well? Specifically the S3 bucket resource.

  • @Ivo_AWS - added to the question

2 Answers
1
Accepted Answer

You can replace the S3 trigger with an EventBridge rule. This is a more modern way of linking an S3 action to a Lambda invocation, and will not trigger this circular dependency. Code in SAM would look like this:

      Events:
        EventBridgeTrigger:
          Type: EventBridgeRule
          Properties:
            EventBusName: default
            Pattern:
              source:
                - aws.s3
              detail-type:
                - Object Created
                - Object Removed
              detail:
                bucket:
                  name:
                    - !Ref photosS3Bucket
profile pictureAWS
answered a month ago
  • @Ivo_AWS - thanks! that worked. I did have to enable event bridge notifications on the s3 bucket in order for it to work...

      photosS3Bucket:
        Type: AWS::S3::Bucket
        Properties:
          BucketEncryption:
            ServerSideEncryptionConfiguration:
              - BucketKeyEnabled: true
          VersioningConfiguration:
            Status: Enabled
          BucketName: !Sub ${SiteName}-photos
          NotificationConfiguration:
            EventBridgeConfiguration:
              EventBridgeEnabled: true
    
    
-1

The circular dependency issue you're experiencing is a common problem when setting up S3 event triggers for Lambda functions using SAM templates. This occurs because the Lambda function depends on the S3 bucket for its event source, while the S3 bucket needs to reference the Lambda function ARN in its notification configuration.

To resolve this circular dependency, you can use a combination of AWS::Serverless::Function and AWS::Lambda::Permission resources instead of relying solely on the Events property of the Lambda function. Here's how you can modify your template to break the circular dependency:

  1. Remove the Events section from your Lambda function definition.

  2. Add a separate AWS::Lambda::Permission resource to grant S3 permission to invoke your Lambda function.

  3. Add a NotificationConfiguration to your S3 bucket resource that references the Lambda function.

Here's an example of how you can restructure your template:

Resources:
  handleS3uploadFunction:
    Type: AWS::Serverless::Function
    Properties:
      Description: !Sub
        - Stack ${AWS::StackName} Function ${ResourceName}
        - ResourceName: handleS3uploadFunction
      CodeUri: backend/
      Handler: src/handlers/handle-s3-upload.s3UploadHandler
      Runtime: nodejs22.x
      MemorySize: 3008
      Timeout: 20
      Tracing: Active
      Environment: {}
      Policies:
        - DynamoDBCrudPolicy:
            TableName: !Ref photoProvTable

  handleS3uploadFunctionPermission:
    Type: AWS::Lambda::Permission
    Properties:
      FunctionName: !Ref handleS3uploadFunction
      Action: lambda:InvokeFunction
      Principal: s3.amazonaws.com
      SourceArn: !Sub arn:aws:s3:::${photosS3Bucket}

  photosS3Bucket:
    Type: AWS::S3::Bucket
    Properties:
      NotificationConfiguration:
        LambdaConfigurations:
          - Event: s3:ObjectCreated:*
            Function: !GetAtt handleS3uploadFunction.Arn
          - Event: s3:ObjectRemoved:*
            Function: !GetAtt handleS3uploadFunction.Arn

This approach breaks the circular dependency by separating the Lambda function definition from its S3 event trigger configuration. The AWS::Lambda::Permission resource grants S3 the necessary permissions to invoke the Lambda function, while the S3 bucket's NotificationConfiguration directly references the Lambda function's ARN.

By implementing these changes, you should be able to resolve the circular dependency error and successfully validate your SAM template.
Sources
What is wrong with my SAM Template? | AWS re:Post
How can I add an existing s3 bucket as a lambda trigger using AWS Cloudformation template? Is it even possible to do so using only CloudFormation template? | AWS re:Post

profile picture
answered a month ago
  • I tried this recommended solution, removing the Events: property of the lambda and switching to the NotificationConfiguration: property of the s3 bucket. Still get the circular reference error

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions