Elastic beanstalk custom scaling policy

Question

Currently i'm working on scaling my elastic beanstalk environment based on memory_used metric.
Based on research i found out that i need to create the scaling policy and attach it to the alarm in the postdeploy hooks, as cloudwatch agent is needed to collect the mem_used_percent metric.
So this is what i've done so far:
1- In the .ebextentions i created a config file for the clowdwatch agent.
2- In the .platform/hooks/postdeploy i created a .sh script to create the policy and clowdwatch alarm

note: I'm using Immutable updates for the elastic beanstalk environment

The issue i have is that when i print the Autoscaling group name i find out that it prints the temporary autoscaling group name not the permanent one so the policy isn't created.

How i can fix it ?

```
#!/bin/bash
sudo yum install jq
INSTANCE="`wget -qO- http://169.254.169.254/latest/meta-data/instance-id`"
Region="`curl --silent http://169.254.169.254/latest/dynamic/instance-identity/document | jq -r .region`"
Account="`curl -s http://169.254.169.254/latest/dynamic/instance-identity/document | jq -r .accountId`"
AutoScalingGroup="`aws autoscaling describe-auto-scaling-instances --instance-ids $INSTANCE --query 'AutoScalingInstances[0].AutoScalingGroupName' --region $Region`"

echo "Printing AutoScalingGroup - Create_scaling"
echo "$AutoScalingGroup"

#Create scale up policy
aws autoscaling put-scaling-policy --auto-scaling-group-name $AutoScalingGroup --policy-name my-simple-scale-up-policy-2 --policy-type SimpleScaling --adjustment-type ChangeInCapacity --scaling-adjustment 1 --cooldown 60 --region $Region

PolicyUp="`aws autoscaling describe-policies --auto-scaling-group-name $AutoScalingGroup --region $Region --query 'ScalingPolicies[?PolicyName==`my-simple-scale-up-policy-2`].PolicyARN | [0] ' `"

#Create alarm on memory and link with scale up policy
aws cloudwatch put-metric-alarm --region=$Region \
--alarm-name 1ScalingMemSpaceUtilization\
--alarm-description "Alarm when MemSpaceUtilization exceeds 80 percent" \
--metric-name mem_used_percent --namespace CWAgent --statistic Average \
--period 300 --threshold 80 --comparison-operator GreaterThanThreshold  \
--dimensions Name=AutoScalingGroupName,Value=$AutoScalingGroup Name=InstanceId,Value=$INSTANCE --evaluation-periods 1 \
--alarm-actions $PolicyUp --unit Percent

```

* The AutoScalingGroup varaible is used to get the name of the current autoscaling group.
* The PolicyUp variable is used to get the ARN of the created policy to be linked with the alarm, so when the alarm is "IN ALARM" state it triggers the scaling policy.

Shahad_C · Answer

Hello, a couple of things

1) For the AutoScalingGroup variable, it would be simpler if you pulled it from IMDS like the 2 other variables by checking the ``` aws:autoscaling:groupName``` tag value
1) This looks like a userdata script, meaning its getting run every time a new instance is launched in the ASG (and thus trying to re-create the scaling policy each time).  You should be creating the scaling policy in the EBExtensions.
1) I recommend using Target Tracking instead of simple scaling.  Its much easier to setup since you don't have to manually define alarms, and you only have to define a single policy that handles both scale-in and scale-out
1) Unless there's a unit defined when the metric is being pushed to CloudWatch, you shouldn't include a unit on the alarm, otherwise the alarm will be looking for a non-existent metric (a metric is a unique combination of Namespace, MetricName, [optional] Dimension(s), [optional] Unit )
1) The dimensions on the alarm aren't setup correctly.  You want to configure the agent to push a single aggregate metric with just AutoScalingGroupName.  You're then scaling off the aggregate values of every instance in the ASG

As a side note, if you're at the point where you're doing this much customization, it might be time to look into migrating off elastic beanstalk to just define everything in CloudFormation natively.  You'll have a lot more control, and the more customization you add to a beanstalk environment the more complex it can be to try and keep track of

Ahmed Elhennawy · Answer

What i've reached so far: 
In the .ebextensions i created 2 config files (The problem i have is that the ScalingOnMemoryAlarmHigh and ScalingOnMemoryAlarmLow are in state "Insufficient data" and i couldn't make them work)

1- 01_cloudwatch.config to configure the cloudwatch agent

```
files:
  "/opt/aws/amazon-cloudwatch-agent/bin/config.json":
    mode: "000600"
    owner: root
    group: root
    content: |
        {
        	"agent": {
        			"metrics_collection_interval": 60,
        			"run_as_user": "root"
        	},
        	"metrics": {
        			"append_dimensions": {
        				"AutoScalingGroupName": "${aws:AutoScalingGroupName}",
        				"InstanceId": "${aws:InstanceId}"
        			},
        			"aggregation_dimensions" : [["AutoScalingGroupName"]],
        			"metrics_collected": {
        			    "mem": {
        			        "measurement": [
                                "mem_used_percent"
                            ],
                            "metrics_collection_interval": 60
                        },
        				"disk": {
        					"measurement": [
        						"used_percent"
        				],
        				"metrics_collection_interval": 60,
        					"resources": [
        							"*"
        				]
        			}
        		}
        	}
        }
```
2-  02_scaling.config to create the cloudwatch alarm to trigger the scaling policy

```
Resources:
  AWSEBCloudwatchAlarmHigh:
    Type: AWS::CloudWatch::Alarm
    Properties:
      AlarmActions: []

AWSEBCloudwatchAlarmLow:
    Type: AWS::CloudWatch::Alarm
    Properties:
      AlarmActions: []

ScalingOnMemoryAlarmHigh:
    Type: AWS::CloudWatch::Alarm
    Properties:
      AlarmDescription: "Scale up when memory used percent > 80"
      Namespace: "CWAgent"
      MetricName: mem_used_percent
      Dimensions:
            - Name: AutoScalingGroupName
              Value:
                Ref: AWSEBAutoScalingGroup`
      Statistic: Average
      Period: 300
      EvaluationPeriods: 1
      Threshold: 80
      ComparisonOperator: GreaterThanThreshold
      AlarmActions:
        - Ref: AWSEBAutoScalingScaleUpPolicy

ScalingOnMemoryAlarmLow:
    Type: AWS::CloudWatch::Alarm
    Properties:
      AlarmDescription: "Scale down when memory used percent < 20"
      Namespace: "CWAgent"
      MetricName: mem_used_percent
      Dimensions:
            - Name: AutoScalingGroupName
              Value:
                Ref: AWSEBAutoScalingGroup
      Statistic: Average
      Period: 300
      EvaluationPeriods: 1
      Threshold: 20
      ComparisonOperator: LessThanThreshold
      AlarmActions:
        - Ref: AWSEBAutoScalingScaleDownPolicy
```

Elastic beanstalk custom scaling policy

Relevant content