Sometimes pulling image from private docker hub registry too slow (ap-south-1)

The slow image pulling from your private Docker Hub registry in the ap-south-1 region could be happening for several reasons, with the interesting behavior that killing and recreating the pod resolves the issue.

This could be due to:

1. Network connectivity issues: Temporary network problems between your EKS nodes and Docker Hub might cause slow downloads that resolve when you restart the pod.

2. Docker Hub rate limits: If you're using a free Docker Hub account, you might be hitting rate limits, especially if there's a delay between pushing the image and deploying it.

3. Resource contention: The initial pod might be experiencing resource limitations that are resolved when you create a new pod.

To improve your image pull times, consider these solutions:

1. Use ECR instead of Docker Hub: Amazon ECR is more tightly integrated with EKS and might provide more reliable image pulls.

2. Reduce image size: Smaller container images pull faster. Consider using minimal base images and optimizing your Dockerfile.

3. Configure image pull retries: Set up your pod specification to retry image pulls before failing.

4. Implement SOCI Parallel Pull Mode: For EKS clusters, this feature can significantly improve pull times by establishing multiple concurrent HTTP connections and downloading layers in chunks.

5. Check image pull secrets: Ensure your credentials for the private registry are correctly configured and not expiring.

6. Consider using a paid Docker Hub account: This can help avoid rate limiting issues.

If you're using EC2 launch type for your nodes, you could also configure the ECS_IMAGE_PULL_BEHAVIOR parameter to "once" or "prefer-cached" to use cached images when available, which can significantly reduce pull times for subsequent pods.

The fact that killing the pod resolves the issue suggests there might be a temporary hang or timeout in the image pulling process that gets reset when you create a new pod.
Sources
EKS creating pod stuck at pulling image | AWS re:Post
Introducing Seekable OCI Parallel Pull mode for Amazon EKS | Containers
EKS Pod Fails to Pull Image from Docker Hub After Manual Trigger in Bitbucket Pipeline | AWS re:Post
Reduce latency when I pull images with Amazon ECS | AWS re:Post
Container image pull behavior - Amazon Elastic Container Service