How to monitor GPU usage in EC2 Inf1 instance?
Hello,
I tried using Inf1 EC2 instance for deploying my ML model. I need to monitor the GPU usage of the ML model. I could find the CPU usage in the aws console, but not gpu usage.
Already tried:
This didn't work. It threw this error
(python3) ubuntu@ip-xxx-mm-yy-zzz:~/tools/GPUCloudWatchMonitor$ python3 gpumon.py
Traceback (most recent call last):
File "gpumon.py", line 146, in <module>
nvmlInit()
File "/home/ubuntu/anaconda3/envs/python3/lib/python3.8/site-packages/pynvml/nvml.py", line 1450, in nvmlInit
nvmlInitWithFlags(0)
File "/home/ubuntu/anaconda3/envs/python3/lib/python3.8/site-packages/pynvml/nvml.py", line 1440, in nvmlInitWithFlags
_nvmlCheckReturn(ret)
File "/home/ubuntu/anaconda3/envs/python3/lib/python3.8/site-packages/pynvml/nvml.py", line 765, in _nvmlCheckReturn
raise NVMLError(ret)
pynvml.nvml.NVMLError_DriverNotLoaded: Driver Not Loaded
Also, nvidia-smi
didn't work
(python3) ubuntu@ip-xxx-mm-yy-zzz:~/tools/GPUCloudWatchMonitor$ nvidia-smi
NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.
Kindly provide some help to monitor GPU usage.
Hello,
EC2 Inf1 instances feature AWS Inferentia chips (high-performance machine learning inference chips designed and built by AWS). In order to monitor usage on these chips, you need to use tools available in the Neuron SDK as opposed to the GPU usage tools that you have tried. A guide to all of the available profiling, debugging, and monitoring tools available for Neuron can be found here: https://awsdocs-neuron.readthedocs-hosted.com/en/latest/neuron-guide/neuron-tools/index.html.
Thanks!
Relevant questions
How to monitor GPU usage in EC2 Inf1 instance?
AWS-User-5413662asked 6 days agoDeploying a Machine Learning Project with django and laravel as a backend.
RazaibAWSasked 7 days agoWhat's the best way to monitor NVIDIA GPU utilization on Linux (Ubuntu) during model training?
Accepted AnswerIoanasked a year agovfio-pci and non metal G4dn EC2 instances
Tarikasked 5 months agoWifi monitor mode with Kali in ec2?
AWS-User-7198048asked a month agoMachine Learning image training on EC2 with GPU
AWS-User-0426098asked 4 months agoWhy is the GPU not working out of the box for Deep learning AMI EC2 instance?
AWS-User-9739481asked 25 days agoSavings Plans Applicability to ML instances
Accepted AnswerAWS-User-5516444asked 2 years agoLimit Monitor EC2 LImits
Accepted Answeralieninvaderasked 3 years agoUse Nvidia GPU in AWS Batch
Nuthinkingasked 2 months ago