HTTP 503 during Rolling Update of managed node group version on EKS with ALB

0

Hello,

I have an EKS cluster with a managed node group, running a web service with an Ingress / ALB (with AWS Load Balancer Controller) in front. On the ALB I set the deregistration_delay.timeout_seconds to 30 seconds and in my K8s web service deployment, I set the terminationGracePeriodSeconds to 70 seconds and I am doing a sleep 40 in the preStop lifecycle hook to achieve zero downtime rollouts/deployments.

This is working great for kubectl rollout restart deployment ... but everytime I update the node group version (Rolling Update in the management console) to the most recent AMI, I get HTTP 503 errors during the rolling update (AWS docs: There are no registered targets in the ALB target group?)

Not sure if it is relevant but the nodes are running Bottlerocket OS 1.15.0 (aws-k8s-1.27) and EKS version is 1.27 (platform version eks.5), the AWS Load Balancer Controller version is v2.6.1 and I also have the elbv2.k8s.aws/pod-readiness-gate-inject=enabled label set in my namespace.

Thank you for any tips on what could be causing this difference between normal deployment rollouts and node group updates and how to achieve zero-downtime for both.

Clemens
已提问 8 个月前172 查看次数
没有答案

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则