EKS error - failed to inspect image

Question

Hey all,
We have a EKS cluster version 1.21 working well for a long time.
Yesterday we started experiencing pods failing to spin up, and then nodes stuck in NotReady state.
The event I could find is:

Failed to inspect image "602401143452.dkr.ecr.eu-west-1.amazonaws.com/amazon-k8s-cni-init:v1.7.5-eksbuild.1": Id or size of image "602401143452.dkr.ecr.eu-west-1.amazonaws.com/amazon-k8s-cni-init:v1.7.5-eksbuild.1" is not set

Why did it happen? 
How should we fix it?

Parthasaradi · Answer

Hello,

Please try this below commands,

**Step 1: SSH into the problematic node and restart Docker**

ssh -i /path/to/your-key.pem ec2-user@your-node-ip

sudo systemctl restart docker

**Step 2: Drain and terminate the problematic nodes**

kubectl get nodes

kubectl drain  --ignore-daemonsets --delete-local-data

aws ec2 terminate-instances --instance-ids

https://repost.aws/knowledge-center/eks-pod-status-troubleshooting

Didier_Durand · Answer

Hi,

Some people face same problem due to incompatibilities between Kubernetes and recent versions of Docker after its upgrade: see https://github.com/moby/moby/issues/47215

```
Until docker-ce v24.0.7, kubernetes image components are successfully run by kubelet. 
With the upgrade of docker-ce to v25.0, kubelet cannot start up the main components of k8s 
with an error stating ID and size unknown

kubelet[21094]: E0125 07:46:51.645034   21094 remote_image.go:94] ImageStatus failed: Id or size 
of image "k8s.gcr.io/kube-proxy:v1.17.12" is not set kubelet[21094]: E0125 07:46:51.645064   
21094 kuberuntime_image.go:85] ImageStatus for image {"k8s.gcr.io/kube-proxy:v1.17.12"} failed: 
Id or size of image "k8s.gcr.io/kube-proxy:v1.17.12" is not set E0125 07:46:51.645109   21094 
kuberuntime_manager.go:809] container start failed: ImageInspectError: Failed to inspect image 
"k8s.gcr.io/kube-proxy:v1.17.12": Id or size of image "k8s.gcr.io/kube-proxy:v1.17.12" is not set 
Error syncing pod 3ed55839-d24d-482a-a2ea-5fa52af9a07a ("kube-proxy-r6kxl_kube-system(
3ed55839-d24d-482a-a2ea-5fa52af9a07a)"), skipping: failed to "StartContainer" for "kube-proxy" 
with ImageInspectError: "Failed 
to inspect image \"k8s.gcr.io/kube-proxy:v1.17.12\": Id or size of image \"k8s.gcr.io/kube-proxy:v1.17.12\" 
is not set"
```

Maybe you are in a similar case?

I guess that you should check if the build process of your container images has changed recently due to the upgrade of some core component.(Docker or other low-level layer)

Best,

Didier

EKS error - failed to inspect image

Relevant content