Want to create g5g instance for EC2

EC2 g5g instances use the arm64 CPU architecture, not the x86 CPU architecture. So you will need to launch your instance with an arm64-compatible AMI.

There are a few Deep Learning AMIs that can support g5g instance type, below are their release notes:

• https://aws.amazon.com/releasenotes/deep-learning-ami-graviton-gpu-pytorch-1-10-ubuntu-20-04/
• https://aws.amazon.com/releasenotes/aws-deep-learning-ami-graviton-gpu-tensorflow-2-6-ubuntu-20-04/
• https://aws.amazon.com/releasenotes/aws-deep-learning-ami-graviton-gpu-cuda-11-4-ubuntu-20-04/

You could probably use one of the above AMIs for your g5g work.

The AMIs available in Marketplace are full of bloatware and are really outdated. For example, Nividia AMI is based on Ubuntu 20.04 and does not even support Cuda 12. And I tried to update that and found myself caught in the chain-dependency problem. After having enough of it I decided to take matters into my own hands. After going through the process I decided to streamline the whole thing and created the magic script that setups the G5g instance. It installs the latest Nvidia Drivers, Cuda Toolkit, cuDNN, and Latest PyTorch as well. And the best part is it is built on top of Amazon Linux 2023. Here is the link to the guide https://mirzabilal.com/deep-learning-with-aws-graviton2-nvidia-tensor-t4g-for-as-low-as-free-with-cuda-12-2-56d8457a6f6d

Disclaimer: I am the author of this article