Unable to retrieve all EC2 instances

0

I'm writing a code to get EC2 On-Demand prices for a particular OS in a determined region (eg. 'Ubuntu Pro' and 'us-east-1') but I only get 661 instances out of the 705 available on the pricing list. Is there a way that I can retrieve all the instances available for said region and OS - with the correct price?

Pricing list:

Enter image description here

Database:

Enter image description here

Python Code:

import boto3
import json
import os
from database import insert_into_table

def ec2_prices(region, op_system, database_name, table_name):
    desired_region = region
    pricing_region = 'us-east-1'

    ec2            = boto3.client('ec2', region_name=desired_region)
    pricing_client = boto3.client('pricing', region_name=pricing_region)

    response       = ec2.describe_instance_type_offerings(LocationType='region')
    instance_types = [instance_type['InstanceType'] for instance_type in response['InstanceTypeOfferings']]
      
    for instance_type in instance_types:
        response = pricing_client.get_products(
            ServiceCode='AmazonEC2',
            Filters=[
                {'Type': 'TERM_MATCH', 'Field': 'instanceType', 'Value': instance_type},
                {'Type': 'TERM_MATCH', 'Field': 'operatingSystem', 'Value': op_system},
            ],
        )
        for product in response['PriceList']:
            product_dict         = json.loads(product)
            attributes           = product_dict.get('product').get('attributes')
            
            instance_region      = attributes.get('regionCode')
            instance_region_name = attributes.get('location')
            instance_memory      = attributes.get('memory')[:-3]
            instance_vcpu        = attributes.get('vcpu')
            instance_os          = attributes.get('operatingSystem')
            tenancy              = attributes.get('tenancy')
            
            if instance_region == desired_region and tenancy == 'Shared':
                terms = product_dict.get('terms', {}).get('OnDemand', {})
  
                for term_key, term_value in terms.items():
                    price_dimensions = term_value['priceDimensions']

                    for _, price_dimension_value in price_dimensions.items():
                        price       = price_dimension_value['pricePerUnit']  
                        description = price_dimension_value['description']                      
                        value       = float(price.get('USD', 0))
                        
                        if value > 0 and f'{op_system} {instance_type}' in description:                            
                            insert_into_table(
                                database_name, 
                                table_name, 
                                instance_type, 
                                value, 
                                instance_region, 
                                instance_vcpu, 
                                instance_memory, 
                                instance_os
                            )

There are some tags (tenancy and description) that are used for retrieving the right price for the instance.

Maria
질문됨 4달 전159회 조회
2개 답변
0

Hi,

Your code looks correct. But, you make 2 important API calls: instance list and pricing info,

Did you check how many answers you get in your for instance_type in instance_types: ? 705 or something different ?

Then, if you have all instance types, the pricing for a given instance may be missing.

So, I would all some counters in your code to see if you get all instance types or not and then check if issue comes with pricing and collect the types for which it's missing to understand.

Best,

Didier

profile pictureAWS
전문가
답변함 4달 전
  • Hello!

    It gets 761 instances - the problem is with the pricing, it's probably missing. If I delete the tenancy flag, it gets 701 instances, but those don't have the correct price.

    Apparently the API can't retrieve all the instance prices available on the Pricing List, but I'm not entirely sure.

0

The output from that API call might be paginated. One thing your code isn't doing is checking for the existence of NextToken in the output - that can be sent to the next API call to continue to get results.

Ref: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ec2/client/describe_instance_type_offerings.html

profile pictureAWS
전문가
답변함 4달 전
  • Unfortunately there's no NextToken parameter in the output. The results are all there, so the API isn't really getting all the instances available in the price list for determined region and OS.

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠