HTTP 503 during Rolling Update of managed node group version on EKS with ALB

0

Hello,

I have an EKS cluster with a managed node group, running a web service with an Ingress / ALB (with AWS Load Balancer Controller) in front. On the ALB I set the deregistration_delay.timeout_seconds to 30 seconds and in my K8s web service deployment, I set the terminationGracePeriodSeconds to 70 seconds and I am doing a sleep 40 in the preStop lifecycle hook to achieve zero downtime rollouts/deployments.

This is working great for kubectl rollout restart deployment ... but everytime I update the node group version (Rolling Update in the management console) to the most recent AMI, I get HTTP 503 errors during the rolling update (AWS docs: There are no registered targets in the ALB target group?)

Not sure if it is relevant but the nodes are running Bottlerocket OS 1.15.0 (aws-k8s-1.27) and EKS version is 1.27 (platform version eks.5), the AWS Load Balancer Controller version is v2.6.1 and I also have the elbv2.k8s.aws/pod-readiness-gate-inject=enabled label set in my namespace.

Thank you for any tips on what could be causing this difference between normal deployment rollouts and node group updates and how to achieve zero-downtime for both.

Clemens
已提問 8 個月前檢視次數 172 次
沒有答案

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南