Can I know the size of spot capacity pools?
Hi, community members,
I am new to AWS cloud service, and I want to use spot GPU instances to train DL models. I wonder can I know the size of spot capacity pools at any time? Since preemption may occurs due to capcaity reason, I want to know whether the change of capacity is available to users.
Thanks a lot!
I am not aware of any way to find out the size of the capacity pools.
But I can make an assessment of the likelihood of a specific instance type being unavailable (or being withdrawn at short notice) by looking at the historic SPOT PRICE movements.
Go to the EC2 console, choose "Spot Requests", and click on "Pricing History". This will give you the historic trend of prices for a specific instance type in that Region. I usually look at the "3 month" trend for that instance type, such as p3.16xlarge.
You can also click on "Spot placement score". This helps you to select optimal Regions or Availability Zones to run workloads that can use multiple instance types.
Hope this helps.
Thanks a lot! From aws doc, I can find the following statement, "When the maximum price per hour for your request exceeds the Spot price, Amazon EC2 fulfills your request if capacity is available." The pricing history can help me know the spot instance price, this is helpful for price biding preemption. But there is also capacity-based preemption. I wonder how the capacity changes since it will determine how many spot instances I can request.
There's no way for customers to see capacity information (other than "did I get a launch failure"). The best ways to have overall fleet availability are:
- Use multiple instances types and AZs. This is easily managed through using an Auto Scaling Group
- Use the CapacityOptimized setting for requesting instances, and listen for Rebalance Notifications to see when a given running instance is at a high risk of being terminated soon. This can be done on the ASG or via IMDS: https://docs.aws.amazon.com/autoscaling/ec2/userguide/capacity-rebalance.html
Thanks a lot!
Spot Placement Score is the most appropriate way to understand how likely it is that a Spot request will succeed in a Region or Availability Zone. (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-placement-score.html)
Can we use spot fleet in cluster config section?asked 10 months ago
ec2 (Spot instances) going from Runing - Initializing to Terminatedasked 2 months ago
ECS Capacity providers best practicesAccepted Answerasked 2 months ago
Can I know the size of spot capacity pools?asked a month ago
Spot fleet request over-provisionedAccepted Answerasked 2 months ago
Hibernating Spot Instances upon interruption in Amazon EKSAccepted Answerasked 2 years ago
Service Quota and Spot Instances vCPUsAccepted Answerasked 2 years ago
ECS + Spot Integration - Multiple ASGs vs SpotFleetAccepted Answerasked 4 years ago
Spot Fleet Instance could not be stoppedasked 22 days ago
Spot persistent requestasked 25 days ago