aws-node Daemonset (AWS EKS v1.21) with strange readiness timeoutSeconds

0

We have here 2 EKS clusters and both sometimes appear in events, readiness Probe failure, from any aws-node Pod. Looking for the Daemonset manifest we have:

          livenessProbe:
            exec:
              command:
                - /app/grpc-health-probe
                - '-addr=:50051'
                - '-connect-timeout=2s'
                - '-rpc-timeout=2s'
            initialDelaySeconds: 60
            timeoutSeconds: 5
            periodSeconds: 10
            successThreshold: 1
            failureThreshold: 3
          readinessProbe:
            exec:
              command:
                - /app/grpc-health-probe
                - '-addr=:50051'
                - '-connect-timeout=2s'
                - '-rpc-timeout=2s'
            initialDelaySeconds: 1
            timeoutSeconds: 1
            periodSeconds: 10
            successThreshold: 1
            failureThreshold: 3

how can you see the livenessProbe exec command it has 2 timeouts, one for connection and the other for rpc, and the probe timeoutSeconds is 5 (easialy would be the amount value of exec commands plus 1 second).

Now looking for readinessProbe we have the same exec command from livenessProbe but the timeoutSeconds is only 1 second.

If you check in the EKS Services probably will not find a service linked with these Pods. So it does not affect any service.

Anyway are error messages that could be out from our logs, that don´t make sense, for now, to have.

I guess a simple fix in readiness timeoutSeconds to 5 (like liveness) in this Deamonset would be enough.

Did anyone have this problem or think of resolving it in this way?

  • One idea I guess is to set de timeoutSeconds of readinessProbe with the same value of livenessProbe (5) because both of them use the same command to check so the timeout should be the same in the first moment. My doubt is if change it, the eksctl utils update-aws-node --cluster=<clusterName> would brake.

profile picture
asked 2 years ago133 views
No Answers

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions