Compatibility issues on DLAMI with aws-neuron-dkms

0

I have setup a DLAMI (both with Deep Learning Base AMI (Amazon Linux 2) Version 57.2 (ID: ami-07d28e42a376a75fe) or Deep Learning Base AMI (Ubuntu 18.04) Version 56.6 (ID: ami-0be3ca7b9b68fa36c)) and machine inf1.6xlarge.

Then I installed everything according to https://awsdocs-neuron.readthedocs-hosted.com/en/v1.17.0/neuron-intro/pytorch-setup/pytorch-quickstart.html and to https://awsdocs-neuron.readthedocs-hosted.com/en/latest/frameworks/torch/torch-neuron/setup/pytorch-install.html#install-neuron-pytorch.

Now when I go into the Jupyter Notebook of this one here https://awsdocs-neuron.readthedocs-hosted.com/en/latest/src/examples/pytorch/resnet50.html it seems to work nicely in the beginning but, at the end I get an error at

model_neuron = torch.jit.load('resnet50_neuron.pt')

saying

2023-Apr-21 11:17:54.0589  8816:8816  ERROR   NRT:nrt_init                                This Neuron Runtime (compatibility id: 6) is not compatible with the installed aws-neuron-dkms package.
2023-Apr-21 11:17:54.0589  8816:8816  ERROR   NRT:nrt_init                                Installed aws-neuron-dkms supports Runtime compatibility id range: 2 - 4.
2023-Apr-21 11:17:54.0589  8816:8816  ERROR   NRT:nrt_init                                Please make sure to upgrade to latest aws-neuron-dkms; for detailed installation instructions visit Neuron documentation.
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[6], line 12
      9 output_cpu = model(image)
     11 # Load the compiled Neuron model
---> 12 model_neuron = torch.jit.load('resnet50_neuron.pt')
     14 # Run inference using the Neuron model
     15 output_neuron = model_neuron(image)

File ~/aws_neuron_venv_pytorch/lib/python3.8/site-packages/torch_neuron/jit_load_wrapper.py:13, in jit_load_wrapper.<locals>.wrapper(*args, **kwargs)
     11 @wraps(func)
     12 def wrapper(*args, **kwargs):
---> 13     script_module = jit_load(*args, **kwargs)
     14     found_neuron_function = False
     15     for node in script_module.graph.nodes():

File ~/aws_neuron_venv_pytorch/lib/python3.8/site-packages/torch/jit/_serialization.py:162, in load(f, map_location, _extra_files)
    160 cu = torch._C.CompilationUnit()
    161 if isinstance(f, str) or isinstance(f, pathlib.Path):
--> 162     cpp_module = torch._C.import_ir_module(cu, str(f), map_location, _extra_files)
    163 else:
    164     cpp_module = torch._C.import_ir_module_from_buffer(
    165         cu, f.read(), map_location, _extra_files
    166     )

RuntimeError: The PyTorch Neuron Runtime could not be initialized. Neuron Driver issues are logged
to your system logs. See the Neuron Runtime's troubleshooting guide for help on this
topic: https://awsdocs-neuron.readthedocs-hosted.com/en/latest/

However, this is how it looks like on my Ubuntu machine

(aws_neuron_venv_pytorch) ubuntu@ip-172-31-13-82:~/aws-neuron-sdk/src/examples/pytorch$ sudo apt-get install aws-neuron-dkms -y
Reading package lists... Done
Building dependency tree       
Reading state information... Done
aws-neuron-dkms is already the newest version (2.3.26.0).
0 upgraded, 0 newly installed, 0 to remove and 17 not upgraded.
(aws_neuron_venv_pytorch) ubuntu@ip-172-31-13-82:~/aws-neuron-sdk/src/examples/pytorch$ sudo apt-get install aws-neuron-tools -y
Reading package lists... Done
Building dependency tree       
Reading state information... Done
aws-neuron-tools is already the newest version (2.1.4.0).
0 upgraded, 0 newly installed, 0 to remove and 17 not upgraded.

Hence, it seems I have the most recent versions up and running. Do you know how to fix that?

1 Answer
0

Hi there. Please try removing the aws-neuron-dkms (no longer used for most customers) package and install aws-neuronx-dkms. In general all packages should include the neuronx naming unless you are working with inf1, in that case you still need some -neuron package, such as torch-neuron

AWS
answered 10 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions