Why does ECS deploy fail ALB healthchecks randomly?

I'm seeing this error in random CDK deploys

Task failed ELB health checks in (target-group arn:aws:elasticloadbalancing:ap-southeast-2:649922175064:targetgroup/....

Basically the first task will fails with the above message and is stopped but ECS will retry launching another task and then everything works. The problem is this adds at least an additional 5 minutes to my deploy time which is not ideal. The only change being deployed is a new ECR image and the task definition pointing to the new image. I don't understand why ECS tasks occasionally fails ELB health checks but it means having to waiting for the failed task to completely stop before ECS will retry to launch another task.

How could we debug the issue?