G5 EC2 instances do not have a proper driver to talk to GPU

I started a G5.xlarge and a 2xlarge instance on ec2. However, when I tried to check on the GPU status via the nvidia-smi command, it errored with: NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.

Why is this happening? G4 instances worked fine though. Seems to be just g5?