HTTP 503 during Rolling Update of managed node group version on EKS with ALB

0

Hello,

I have an EKS cluster with a managed node group, running a web service with an Ingress / ALB (with AWS Load Balancer Controller) in front. On the ALB I set the deregistration_delay.timeout_seconds to 30 seconds and in my K8s web service deployment, I set the terminationGracePeriodSeconds to 70 seconds and I am doing a sleep 40 in the preStop lifecycle hook to achieve zero downtime rollouts/deployments.

This is working great for kubectl rollout restart deployment ... but everytime I update the node group version (Rolling Update in the management console) to the most recent AMI, I get HTTP 503 errors during the rolling update (AWS docs: There are no registered targets in the ALB target group?)

Not sure if it is relevant but the nodes are running Bottlerocket OS 1.15.0 (aws-k8s-1.27) and EKS version is 1.27 (platform version eks.5), the AWS Load Balancer Controller version is v2.6.1 and I also have the elbv2.k8s.aws/pod-readiness-gate-inject=enabled label set in my namespace.

Thank you for any tips on what could be causing this difference between normal deployment rollouts and node group updates and how to achieve zero-downtime for both.

Clemens
preguntada hace 8 meses172 visualizaciones
No hay respuestas

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas