SageMaker autoscaling doesn't work as expected

I've configured a model for realtime inference and it's working correctly. I'm now trying to apply an autoscaling policy and using boto3 library to put the scaling policy. The code that I've used:

import pprint
import boto3
from sagemaker import get_execution_role
import sagemaker
import json

endpoint_name = "a-deployed-endpoint"
asg_client = boto3.client('application-autoscaling')
resource_id='endpoint/' + endpoint_name + '/variant/' + 'AllTraffic'
asg_client.register_scalable_target(
    ServiceNamespace='sagemaker', #
    ResourceId=resource_id,
    ScalableDimension='sagemaker:variant:DesiredInstanceCount',
    MinCapacity=1,
    MaxCapacity=5
)

response = asg_client.put_scaling_policy(
    PolicyName='SageMakerEndpointInvocationScalingPolicy',
    ServiceNamespace='sagemaker',  
    ResourceId=resource_id,
    ScalableDimension='sagemaker:variant:DesiredInstanceCount',
    PolicyType='TargetTrackingScaling',
    TargetTrackingScalingPolicyConfiguration={
        'TargetValue': 20.0,
        'PredefinedMetricSpecification': {
            'PredefinedMetricType': 'SageMakerVariantInvocationsPerInstance',
        },
        'ScaleInCooldown': 60, 
        'ScaleOutCooldown': 10 
    }
)

To confirm the policy has been configured correctly, I've double-checked with describe_scaling_policies method from boto3 sdk and on AWS console.

With describe_scaling_policies method, we have:

res = asg_client.describe_scaling_policies(
    PolicyNames=['SageMakerEndpointInvocationScalingPolicy'],
    ServiceNamespace='sagemaker', # The namespace of the AWS service that provides the resource. 
    ResourceId=resource_id, # Endpoint name 
    ScalableDimension='sagemaker:variant:DesiredInstanceCount', # SageMaker supports only Instance Count
)
print(res)

###### print's result ######
{
    'ScalingPolicies': [
        {
            'PolicyARN': 
'arn:aws:autoscaling:ap-southeast-1:***:scalingPolicy:c97d566f-50f0-4ac4-ac23-765a815095e9:resource/sagema
ker/endpoint/jets-gpu-endpoint/variant/AllTraffic:policyName/SageMakerEndpointInvocationScalingPolicy',
            'PolicyName': 'SageMakerEndpointInvocationScalingPolicy',
            'ServiceNamespace': 'sagemaker',
            'ResourceId': 'endpoint/jets-gpu-endpoint/variant/AllTraffic',
            'ScalableDimension': 'sagemaker:variant:DesiredInstanceCount',
            'PolicyType': 'TargetTrackingScaling',
            'TargetTrackingScalingPolicyConfiguration': {
                'TargetValue': 20.0,
                'PredefinedMetricSpecification': {
                    'PredefinedMetricType': 'SageMakerVariantInvocationsPerInstance'
                },
                'ScaleOutCooldown': 10,
                'ScaleInCooldown': 60
            },
            'Alarms': [
                {
                    'AlarmName': 
'TargetTracking-endpoint/jets-gpu-endpoint/variant/AllTraffic-AlarmHigh-bc06a793-369a-4515-8a7f-3cb5127455b6',
                    'AlarmARN': 
'arn:aws:cloudwatch:ap-southeast-1:***:alarm:TargetTracking-endpoint/jets-gpu-endpoint/variant/AllTraffic-
AlarmHigh-bc06a793-369a-4515-8a7f-3cb5127455b6'
                },
                {
                    'AlarmName': 
'TargetTracking-endpoint/jets-gpu-endpoint/variant/AllTraffic-AlarmLow-9889edca-45f2-4f73-9f71-76cacf7b5882',
                    'AlarmARN': 
'arn:aws:cloudwatch:ap-southeast-1:***:alarm:TargetTracking-endpoint/jets-gpu-endpoint/variant/AllTraffic-
AlarmLow-9889edca-45f2-4f73-9f71-76cacf7b5882'
                }
            ],
            'CreationTime': datetime.datetime(2023, 5, 26, 7, 3, 12, 714000, tzinfo=tzlocal())
        }
    ],
    'ResponseMetadata': {
        'RequestId': '56238668-ee84-4319-b2a9-8448cac98358',
        'HTTPStatusCode': 200,
        'HTTPHeaders': {
            'x-amzn-requestid': '56238668-ee84-4319-b2a9-8448cac98358',
            'content-type': 'application/x-amz-json-1.1',
            'content-length': '1341',
            'date': 'Fri, 26 May 2023 07:03:14 GMT'
        },
        'RetryAttempts': 0
    }
}

With AWS console:

Everything looks correct, right ? So I head to some load tests to make sure that the autoscaling works fine (for more information, I've used locust library to carry out the test). After 5 minutes, the endpoint starts encountering a bottleneck and also the high-traffic cloudwatch raises an alarm as the image below:

The weird thing here is after 20 minutes since the cloudwatch raised an alarm, I don't see the endpoint scales out instances as expected, although I've tried to use a low target threshold and small scale out time. So, any clue or suggesstion to make the endpoint scalable?