boto3 ecs.describe_task call returns task missing

Question

I'm trying to use a boto3 ECS waiter to wait on a Fargate ECS task to complete. The vast majority of the time the waiter works as expected (waits for the task to reach the STOPPED status). However, sporadically the waiter will return a failure because a task is marked as missing. However, I can find the task itself in the cluster and cloudwatch logs for the task. When I first encountered this, I switched to using boto3 [`ecs.describe_tasks`](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ecs.html#ECS.Client.describe_tasks) method to see if I could get more information about what was happening. When the above situation occurs, descirbe_tasks returns something like:
```
{'tasks': [], 'failures': [\{'arn': 'arn:aws:ecs:us-west-2:21234567891011:task/something-something/dsfsadfasdhfasjklhdfkdsajhf', 'reason': 'MISSING'}], 'ResponseMetadata': \{'RequestId': sdkfjaskdjfhaskdjfhasd', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'sdkjfhsdkajhfksadhkjsadf', 'content-type': 'application/x-amz-json-1.1', 'content-length': '145', 'date': 'Fri, 06 May 2022 08:36:11 GMT'}, 'RetryAttempts': 0}}
```

I've looked at the [AWS Docs ](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/api_failures_messages.html) and the none of the scenarios outlined for `reason: MISSING` apply in my circumstance. I'm passing my cluster name as an argument to the call as well. Since this happens intermittently its difficult to troubleshoot. What does the MISSING status mean? What are the reasons why an API call to check on task status would return missing when the task exists?

Answer

I understand that you are trying to setup a mechanism where a boto3 ECS waiter function to wait until a Fargate task is Stopped. So whenever you are running “ecs.describe_tasks” boto3 method and randomly running in to an issue where the task with a specific task Id is reported as MISSING. Now, I assume that you are essentially trying to obtain Describe task API output to find out if the task is stopped(completed) or to find out if it has failed. Now, when a task is stopped, the details of the task are available for review by Describe task only for a certain amount of time.  At the moment, ECS does not have a fixed time as to when the Stopped task details are reaped(details unavailable for retrieval) using the Describetask API.

For example, it is expected that a DescribeTask API can return a response that the task is MISSING even shortly after the task is Stopped. Because the task details in some cases/scenarios can be removed(for retrieval) by ECS as early as minutes and sometimes can stay for an hour(approximately).

To answer you question as to what does the MISSING status mean?

In addition to the obvious reasons where the task ID/region/Cluster name are incorrect. You could also see the MISSING status for a task ID when that specific Task has been stopped and ECS had reaped this stopped task. Based on the information that you have shared with me and considering that the issue is intermittent, this could be main reason you are observing some tasks showing up as MISSING status.

Now I understand that you might be able to view these stopped tasks in the console which ideally you should not be able to view if the Describe task API call is showing the task’s status as MISSING. In such case, I would suggest that you open a support case with AWS Premium Support to troubleshoot the issue further as we would need cluster/task related information.

boto3 ecs.describe_task call returns task missing

Relevant questions

ECS Fargate Task in EventBridge has ResourceInitializationError

Charging operate Task for execution time

EventBus Rule Target ECS Fargate Task - Unable to invoke set version

Unable to override taskRoleArn when running ECS task from Lambda

boto3 ecs.describe_task call returns task missing

Fargate: how to stop task after finishing

ECS fails to remove a task from the load balancer target group?

What needs to be done to make event bridge invoke a fargate task when file added to s3

defining the name of task definition json to run ecs task in github actions

ECS Task Groups