sagemaker custom kernel

Question

Hi all

I registered a custom kernel for python 3.13 and pytorch latest version as follows in a terminal.

1. conda create -n p313_torch python==3.13
2. conda init
3. source activate p313_torch 
4. conda install -c conda-forge gcc_linux-64 gxx_linux-64
5. pip install torch torchvision torchaudio
6. conda deactivate
7. python -m ipykernel install --user --name 'p313_torch' --display-name 'p313_torch'

Then I could see "p313_torch" from the kernel list.

But when I checked the python and torch version in a Jupyter cell using the kernel, they are not found. The followings are my commands that I used and results.

```
##
!python --version
Python 3.10.19

##
import sys
print(sys.version)
print(sys.executable)

3.12.12 | packaged by conda-forge | (main, Oct 22 2025, 23:25:55) [GCC 14.3.0]
/home/ec2-user/anaconda3/bin/python

##
import torch
print(torch.__version__)

ModuleNotFoundError: No module named 'torch'
```

What's happening here?

Thanks.

re:Post Agent · Answer

It appears there's a mismatch between the kernel you created and the environment that's actually being used when you run your code.

When you created your custom kernel with `ipykernel install`, it should have linked to your conda environment (p313_torch), but the output shows that your notebook is still using Python 3.10.19 or 3.12.12 from the Anaconda installation path, not Python 3.13 from your custom environment.

This discrepancy could be happening for a few reasons:

1. In SageMaker notebook instances, kernels are separate conda environments. When you restart a notebook instance, SageMaker refreshes the default environments, which might have overwritten your custom kernel configuration.

2. To make custom kernels persist in SageMaker, you need to install them on the notebook instance's Amazon EBS volume and use lifecycle configuration scripts (both on-create and on-start scripts) to ensure they remain available after restarts.

3. The `!python` command in Jupyter cells uses the system Python, not necessarily the kernel's Python. That's why you're seeing Python 3.10.19 when using the shell command.

4. The `sys.executable` output shows you're using Python from the Anaconda installation path, not from your custom environment.

To fix this issue:
- Make sure your custom environment is properly registered with Jupyter
- Consider using lifecycle configuration scripts to ensure your custom kernel persists
- Verify the kernel is actually selected in your notebook (check the top right corner of your notebook)
- Try using `import sys; sys.executable` to confirm which Python interpreter is being used
- For package installations, use `%pip install` or `%conda install` magic commands instead of shell commands to ensure packages are installed in the current kernel's environment  
**Sources**  
[Install External Libraries and Kernels in Amazon SageMaker Studio Classic - Amazon SageMaker AI](https://docs.aws.amazon.com/sagemaker/latest/dg/studio-notebooks-add-external.html)  
[External library and kernel installation - Amazon SageMaker AI](https://docs.aws.amazon.com/sagemaker/latest/dg/nbi-add-external.html)  
[Amazon Linux 2 notebook instances - Amazon SageMaker AI](https://docs.aws.amazon.com/sagemaker/latest/dg/nbi-al2.html)

Riku_Kobayashi · Answer

Hello.

I'm not very familiar with Conda virtual environments, but I think it might be pointless to run "python -m ipykernel install" after running "conda deactivate".  
So, try switching the order of commands 7 and 6 and executing them.  
The "conda deactivate" command is used to exit an active Conda virtual environment.  
https://docs.conda.io/projects/conda/en/stable/user-guide/tasks/manage-environments.html#deactivating-an-environment

sagemaker custom kernel

Add your answer

Relevant content