EKS Fargate Nodes ami / kubelet version stuck?
I may be missing something, but it seems our fargate nodes aren't updating their AMI and Kubelet version. Fargate notes are 4.14.275-207.503.amzn2.x86_64 / v1.20.7-eks-135321, but our EC2 nodes are on 5.4.188-104.359.amzn2.x86_64 / v1.20.11-eks-f17b81. We are able to update EC2 nodes as the new versions become available and we are notified. The fargate nodes are recycled often as we deploy fairly regularly.
It does seem we must have received SOME update as these clusters have been deployed since 2021, and 1.20.11 was released in March?
Does anyone know why our fargate nodes won't update?
It is consistent across the board on all clusters and there are no eviction errors.
We are not on a technical support tier, so I won't be opening a ticket to spend 1000 dollars on this issue.
Sorry but lack of any insight into your account, it could be hard to give you specific directions. Fargate is supposed to update the AMIs, kubelet and agents automatically. It seems specifically the AMI is few versions behind. If your K8s versions are compatible (which it seems to be the case) the best approach is to get support to see if there is any service related issues. You will not be spending 1000 dollars on this. If you have an account team contact, please reach out to them too.
First about the obvious - is this behavior universal cross the board? are all Fargate pods stuck at a lower version? If not, have you check and see if there are any eviction errors?
If as you suggested you recycle pod fairly frequently and you observed this issue consistently, I would recommend you submit a support ticket so our support engineers can take a deeper look.
In case anyone is looking for an answer some day: "We do not have an ETA on when we will migrate Fargate into kernel 5.x, but kubelet will have a faster and more frequent release cycle. the release cycle for minor versions on Fargate is slower than on EKS AMIs, hence the version might be behind, for example for 1.21, Fargate nodes are on 1.21.2 while EKS AMI is on 1.21.5. If there is a critical security vulnerability on a minor version of the kubelet, we do provision that release as soon as it is discovered, so rest assured that the changes between the minor versions are not critical or security related if there is a difference between both."
kube-proxy failing after update to 1.16+asked 7 months ago
Programatically retrieve recommended AMI id for GPU EKS nodesasked a year ago
Viewing EKS 'fargate-scheduler' logsasked 3 months ago
Encountering "PLEG is not healthy" error on EKS nodesasked 13 days ago
EFS volume in Batch job running on Fargateasked 3 months ago
EBS Unoptimized Productasked 7 days ago
How can I direct kubelet on EKS to contact private VPC endpoints?asked 2 years ago
Implications of automatic EKS Platform version updateasked 2 months ago
EKS Fargate Nodes ami / kubelet version stuck?asked 21 days ago
Mount options for EFS on Fargateasked 3 months ago