cannot Launch instance (kernel) with custom Sagemaker Image

1

The custom image has been used for a long time.

For some reason, when I created new instance by using custom image from sagemaker studio, I keep get "Failed to start kernel" error.

Same image could work fine in the past.

Also I noticed that if failed first, and try with other instance, it could work. Some times if you kept trying on same instance there was a chance worked as well.

I hope sagemaker team aware of this issue.

asked 8 months ago292 views
4 Answers
1

I'm experiencing the same error. My image has also been used for months without any problems. It started yesterday.

Also, cloudwatch logs don't show any errors.

As a temporary fix, you can start an instance with a default kernel (data science 3.0) and then switch to your custom image.

Gonzalo
answered 8 months ago
  • Thanks, this works :)

    But for some instance type (by default, only allow 1 running app per domain), I cannot use this way to switch because after I created one by default kernel, it won't let you create second one with custom image.

    Hope Sagemaker team notice this issue and implement the fix asap.

1

Enter image description here

answered 8 months ago
0
Accepted Answer

I have been worked with Sagemaker support and they implemented the changes, the instance should be created correctly with custom images.

during the issue, @Gonzalo's temp solution was working perfectly

answered 8 months ago
0

I also tested running the notebook from "notebook job". Looks like the kernel from job scheduler can start kernel without custom image without error.

answered 8 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions