跳至內容

Autoscaler Actions by "user request"

0

We have an autoscaler which scales in and out based on policies such as NetworkIn, Healthy Host, etc. We also have some scheduled actions set up. All these actions are properly logged and can be identified in the Actions Log.

However, I keep seeing Scale-In and Scale-Out actions logged "in response to a user request" (see below example). What can cause these and how can I pinpoint them? I promise none of us is manually changing desired or minimum settings)

At 2024-07-10T23:51:48Z instance i-0c705997aaa028d42 was taken out of service in response to a user request, shrinking the capacity from 5 to 4.

已提問 2 年前檢視次數 943 次
3 個答案
1

This is most often from the TerminateInstanceInAutoScalingGroup API. Its commonly used by some services (such as EKS, Batch, or CloudFormation), as well as some 3rd party applications which are managing instances in an ASG. As the other replies recommended, go to CloudTrail and search around the start time of one of these activities by either: ReadOnly=False or EventSource=autoscaling.amazonaws.com

You'll want to look at the UserAgent of the call in cloudtrail to see where its coming from

專家
已回答 2 年前
  • thank you, I have activated cloudtrail now. We found some issues with our load balancer health checks which target this autoscaler group, could it be that the load balancer does these "user requests" to make sure we have healthy instances in the autoscaler?

  • No, failed ELB healthchecks would have an explicit message in the Activity History. Its possible some other system is noticing the failed ELB healthchecks and sending these termination requests to AutoScaling, but the ELB itself won't do that

    You should be able to see 90 days of past CloudTrail events by default, even without a Trail setup to store them externally. You can see these past events on the CloudTrail console, or via the lookup-events API

1

Hello.

If the change was made by a user, I think you could check which IAM user changed the EC2 capacity by checking the CloudTrail event history.
https://docs.aws.amazon.com/awscloudtrail/latest/userguide/view-cloudtrail-events-console.html

專家
已回答 2 年前
專家
已審閱 2 年前
專家
已審閱 2 年前
0

Hii

Check these troubleshoot steps to resolve issue:

  • Identify Specific Actions: Look for entries in the Actions Log with "in response to a user request" next to the scaling action (scale-in or scale-out).
  • Review User Activity: Check if any users made scaling requests through the autoscaler's user interface or API during the logged times..

Consider Integrations:

  • Do you have any integrations with other tools that might trigger scaling actions?
  • Investigate if these integrations can initiate scaling and might be misidentified as user requests.
  • Check Scheduled Actions: Double-check your scheduled scaling actions to ensure they aren't causing these events

.

專家
已回答 2 年前
專家
已審閱 2 年前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。