What needs to be done to make event bridge invoke a fargate task when file added to s3

I have continued to investigate - in case anyone else sees this issue. It seems to be us needing to add AmazonECS_FullAccess to the event bridge role. This feels a little like a sledgehammer to crack a nut, but it at least invokes the ECS now. We will come back to this and scale back permissions. If I find a better set of permissions, I will update this post.

I had a similar issue. I was setting up Eventbridge Rule to trigger an ECS Fargate Task when I put a file into an S3 Bucket. I was doing this all through CloudFormation

When I went to Old ECS Experience UI under Scheduled Tasks I could see my tasked scheduled by EventBridge to run on pattern matching.

I then found this page debug page:

• How do I troubleshoot issues related to scheduled tasks in Amazon ECS?

I installed CloudWatch Logs for EventBridge using the following template here:

• https://serverlessland.com/patterns/eventbridge-cloudwatch

This was showing me that my Event Rule was being triggered but I could already tell because under the Event RuleMonitoring tab. So not very useful.

I went to CloudTrail and matching on Event Name I searched for RunTask and nothing came up. So that means its a permissions issue.

When you create an EventBridge Rule in CloudFormation the documentation says theRoleArn is required which actually not true (I discovered its optional in a Github Issue for CloudFormation). I thought the first RoleArn was the CloudWatch Events Role that needed permissions to execute the task.

EventBridgeRule:
  Type: AWS::Events::Rule
  Properties: 
    RoleArn: !GetAtt RuleRole.Arn   
    Targets: 
      - Id: !Sub ${AWS::StackName}TriggerRule
        RoleArn: !GetAtt TaskRole.Arn        
        EcsParameters:
          EnableExecuteCommand: true

CFN template simplified to show both Role fields

However... when I edited my EventBridge Rule in the AWS Console under Targets I could see that my TaskRole was being set but the UI indicated that this role was for CloudWatch Events to be able to execute the task.

So I removed the RoleArn in my CFN template and changed the Target's RoleArn to the Role for the CloudWatch Events Rule.

EventBridgeRule:
  Type: AWS::Events::Rule
  Properties: 
    Targets: 
      - Arn:    
        Id: !Sub ${AWS::StackName}TriggerRule
        RoleArn: !GetAtt RuleRole.Arn        
        EcsParameters:
          EnableExecuteCommand: true

If you're wondering what the permissions need to be there is a page here:

Amazon ECS CloudWatch Events IAM Role

The docs says that a role will be create if you create a Scheduled Task, and I tried to just manually schedule a task to generate it so I could copy its RoleArn but it never generated so I had to just create the expected Policy myself in my CloudFormation Template

If it helps anybody here's my full CFN template

# https://aws.amazon.com/premiumsupport/knowledge-center/ecs-scheduled-task-issues/
# https://docs.aws.amazon.com/AmazonECS/latest/developerguide/CWE_IAM_role.html
AWSTemplateFormatVersion: 2010-09-09
Parameters:
  ClusterStack:
    Type: String
  RepositoryUri:
    Type: String
  SubnetList:
    Type: List<AWS::EC2::Subnet::Id>
    Description: Subnets for the Service
Resources:
  LogGroup:
    Type: AWS::Logs::LogGroup
    Properties:
      LogGroupName: /ts/frontends/eventbridge      
  LogGroupForEventsPolicy:
    Type: AWS::Logs::ResourcePolicy
    Properties:
      PolicyName: EventBridgeToCWLogsPolicy
      PolicyDocument: !Sub >
        {
          "Version": "2012-10-17",
          "Statement": [
            {
              "Sid": "EventBridgetoCWLogsCreateLogStreamPolicy",
              "Effect": "Allow",
              "Principal": { "Service": [ "events.amazonaws.com" ] },
              "Action": [ "logs:CreateLogStream" ],
              "Resource": [ "${LogGroup.Arn}" ]
            },
            {
              "Sid": "EventBridgetoCWLogsPutLogEventsPolicy",
              "Effect": "Allow",
              "Principal": { "Service": [ "events.amazonaws.com" ] },
              "Action": [ "logs:PutLogEvents" ],
              "Resource": [ "${EventBridgeRule.Arn}" ],
              "Condition": { "ArnEquals": {"AWS:SourceArn": "${EventBridgeRule.Arn}"} }
            }
          ]
        }      
  EventBridgeRule:
    Type: AWS::Events::Rule
    Properties: 
      Name: !Sub ${AWS::StackName}TriggerRule
      Description: S3 to trigger ECS task for ts-ui-static-task
      EventPattern: >
        {
          "source": ["aws.s3"],
          "detail-type": ["AWS API Call via CloudTrail"],
          "detail": {
            "eventSource": ["s3.amazonaws.com"],
            "eventName": ["PutObject"],
            "requestParameters": {
              "bucketName": ["ts-frontends"],
              "key": [{"prefix": "manifests/" }]
            }    
          }
        }         
      #RoleArn: !GetAtt RuleRole.Arn   
      State: ENABLED
      Targets: 
        - Arn:
            Fn::ImportValue:
              !Sub ${ClusterStack}ClusterArn        
          Id: !Sub ${AWS::StackName}TriggerRule
          RoleArn: !GetAtt RuleRole.Arn        
          EcsParameters:
            EnableExecuteCommand: true
            Group: ts-ui-static
            LaunchType: FARGATE
            PlatformVersion: LATEST
            TaskDefinitionArn: !Ref TaskDefintion
            TaskCount: 1
            TagList:
              - Key: 'manifest_s3_key'
                Value: '$.detail.object.key'
            NetworkConfiguration:
              AwsVpcConfiguration:
                AssignPublicIp: ENABLED
                Subnets: !Ref SubnetList
                SecurityGroups:
                  - !Ref TaskSecurityGroup
        - Arn: !GetAtt LogGroup.Arn
          Id: !Sub ${AWS::StackName}LogGroup                        
  TaskSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      VpcId:
        Fn::ImportValue:
          !Sub ${ClusterStack}VpcId
      GroupDescription: "TS UI Static SG"
  CloudWatchLogsGroup:
    Type: AWS::Logs::LogGroup
    Properties:
      LogGroupName: !Ref AWS::StackName
      RetentionInDays: 365
  TaskDefintion:
    Type: AWS::ECS::TaskDefinition
    Properties:
      Cpu: "1024"
      Memory: "2048"
      NetworkMode: awsvpc
      ExecutionRoleArn: !GetAtt ExecutionRole.Arn
      TaskRoleArn: !GetAtt TaskRole.Arn
      ContainerDefinitions:
        - Name: app
          Image: !Ref RepositoryUri
          Cpu: 1024
          Memory: 2048
          Essential: true
          LogConfiguration:
            LogDriver: awslogs
            Options:
              awslogs-group: !Ref AWS::StackName
              awslogs-region: !Ref AWS::Region
              awslogs-stream-prefix: app
  ExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
        - Action: ['sts:AssumeRole']
          Effect: Allow
          Principal:
            Service: [ecs-tasks.amazonaws.com]
        Version: '2012-10-17'
      Path: /
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
  RuleRole:
    Type: AWS::IAM::Role
    Properties:
      Path: /
      AssumeRolePolicyDocument:
        Statement:
        - Action: ['sts:AssumeRole']
          Effect: Allow
          Principal:
            Service: [events.amazonaws.com]
        Version: '2012-10-17'
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
      Policies:
        - PolicyName: !Sub ${AWS::StackName}SSMPolicy
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
            - Effect: Allow
              Action:
              - ecs:RunTask
              Resource:
                - "*"
            - Effect: Allow
              Action: iam:PassRole
              Resource:
              - "*"
              Condition:
                StringLike:
                  iam:PassedToService: ecs-tasks.amazonaws.com
  TaskRole:
    Type: AWS::IAM::Role
    Properties:
      Path: /
      AssumeRolePolicyDocument:
        Statement:
        - Action: ['sts:AssumeRole']
          Effect: Allow
          Principal:
            Service: [ecs-tasks.amazonaws.com]
        Version: '2012-10-17'
      Policies:
        - PolicyName: !Sub ${AWS::StackName}SSMPolicy
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Action:
                - ssmmessages:CreateControlChannel
                - ssmmessages:CreateDataChannel
                - ssmmessages:OpenControlChannel
                - ssmmessages:OpenDataChannel
                Effect: Allow
                Resource: '*'
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/CloudWatchFullAccess
        - arn:aws:iam::aws:policy/AWSXRayDaemonWriteAccess
        - arn:aws:iam::aws:policy/AWSAppMeshEnvoyAccess

Refer attached document which contains a tutorial to trigger an ECS task when a S3 file is uploaded.

https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/CloudWatch-Events-tutorial-ECS.html