How to use EC2 instances with multiple GPU instances

Question

I'm trying to train my ml model on mutliple gpus through a EC2 instance, however I'm slightly confused about how to do this. AWS granted me access for 8 vCPUs, however I need at least 96 to instantiate an instance with 2 GPUs. This makes me think I am going about this wrong and am looking for guidance on how to train on multigpu systems through aws.

Answer

Hello!
 
In this scenario, you will want to use Accelerated Computing Instance types such as P3 or G4 instances (to name just two). You can find out more here: https://docs.aws.amazon.com/dlami/latest/devguide/gpu.html
 
Please be aware, though, that not all instance types may be available in any one region. If you do not see any of the recommended instances in your default region, try switching to a different one as they could be available there.

You can double-check which instance types are available in your region or AZ through the describe-instance-type-offerings command on the AWS CLI.
Here's an example command to do just that (based on https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-instance-type-offerings.html): 
    
    aws ec2 describe-instance-type-offerings \
    --location-type availability-zone \
    --filters Name=location,Values=your-az\
    --region your-region

Hope this helps!

How to use EC2 instances with multiple GPU instances

Relevant content