Why does inference on Amazon SageMaker Studio notebook take longer time than Google Colab to complete?

Question

| Amazon SageMaker Studio Notebook | Google Colab |
| --- | --- |
| ml.g4dn.xlarge | GPU accelerator |

Based on the above specifications that was set up for Amazon SageMaker Studio Notebook and Google Colab, it shows that the environments have similarly hardware resources used, particularly Tesla T4 GPU.

However, when running object tracking frame-by-frame inference workload with the **same exact .ipynb code** on both platforms, the results show almost 4x difference in terms of time taken to complete.

For e.g. a 13mins video:
Amazon SageMaker Studio Notebook - took approx. 2 hours
Google Colab - took approx. 30mins

Time taken to process individual frames were also logged but did not see anything abnormal.

Accepted Answer

If the logic for logging of individual frames is implemented and displayed on the the Studio UI, it would have been the cause of the "degraded" performance on Amazon SageMaker Studio.

As the actual inference is done in the backend, every frame to be logged has to send back to the Studio UI to be displayed. The volume of the logs could have been too much that the Studio UI could not display them out quick enough when they are being transported back from the backend to the frontend, hence the delay. So in actual fact, the inference was actually completed. Do expect the time taken for the inference to be completed to be the same as Google Colab and (if possible) check the output directory for the output generated.

Workaround: to reduce the number of logs printed on Studio UI. Instead of logging it frame-by-frame, do so every 50 or 100 frames.

A ticket has been raised and this bug in the Studio UI should see a fix. Any updates would be posted here.

Why does inference on Amazon SageMaker Studio notebook take longer time than Google Colab to complete?

Relevant content