GPU EC2 performance

Based on your requirements and constraints, the best option for improving GPU performance for your Nvidia Isaac Simulator workload would be to consider upgrading to Amazon EC2 G6e instances.

G6e instances are powered by NVIDIA L40S Tensor Core GPUs, which offer significant improvements over the A10G GPUs found in G5 instances. Specifically, G6e instances provide:

1. 2x higher GPU memory (48 GB per GPU compared to 24 GB on G5)
2. 2.9x faster GPU memory bandwidth compared to G6 instances (which are already an improvement over G5)
3. Up to 2.5x better overall performance compared to G5 instances

These improvements should address the performance concerns raised by your developers, particularly in the GPU area. The increased GPU memory and bandwidth can be especially beneficial for complex simulations like those run in Nvidia Isaac Simulator.

G6e instances can be configured with up to 8 NVIDIA L40S GPUs, allowing for significant scaling of your workloads if needed. They also come with other high-performance components, including third-generation AMD EPYC processors, up to 400 Gbps of network bandwidth, and up to 7.6 TB of local NVMe SSD storage.

Given that you're currently using a G5.4xlarge, you might want to start with a comparable G6e instance size and scale up as needed. The exact instance size will depend on your specific requirements, but this upgrade should provide a substantial performance boost for your Nvidia Isaac Simulator workloads without moving to the A100 or H100 chips that aren't supported by your software.
Sources
Amazon EC2 G6e Instances | Amazon Web Services
Amazon EC2 G5 Instances | Amazon Web Services