Questions tagged with AWS Neuron

AWS Neuron is a software development kit (SDK) for running machine learning inference using AWS Inferentia chips.

Content language: English

Select tags to filter
Sort by most recent

Browse through the questions and answers listed below or filter and sort to narrow down your results.

33 results
Hi, I'm trying to run the neuron-device-plugin in our EKS cluster (following https://awsdocs-neuron.readthedocs-hosted.com/en/latest/containers/tutorials/k8s-setup.html) to run inf2.48xlarge nodes. ...
2
answers
0
votes
65
views
asked 3 months ago
I want to test Neuron on an AWS trn1.32xlarge instance using SDK version 2.20 and a PyTorch 2.1 Deep Learning AMI. We've encountered an error with the 'ResizeBilinear' operation, which isn't supported...
2
answers
0
votes
71
views
AWS
asked 3 months ago
Please, guide me how to install neuron runtime on ubuntu 22.04? I always get cannot locate collectives after running this "sudo apt-get install aws-neuronx-collectives=2.* -y" I have WSL Ubuntu 22.04....
1
answers
0
votes
410
views
asked 7 months ago
Hello, i'm using **pytorch-inference:2.0.1-gpu-py310-cu118-ubuntu20.04-sagemaker** AMI and following this guide https://awsdocs-neuron.readthedocs-hosted.com/en/latest/general/setup/neuron-setup/pyto...
1
answers
0
votes
644
views
asked a year ago
hello, i'm using **pytorch-inference:2.0.1-gpu-py310-cu118-ubuntu20.04-sagemaker** image to run the service i'd like to switch to INF2 instance. ~~I think i can try to use **pytorch-inference-neuro...
1
answers
0
votes
744
views
asked a year ago
Hi, One big difference I've noticed between the inf1 and inf2 development experience is that with inf2, we are not able to compile models outside of inf2 instances (unlike inf1 where we were able to...
2
answers
0
votes
391
views
asked a year ago
Which version of torch_neuronx is used in the example? This is asked because neither the API Reference at [aws website](https://awsdocs-neuron.readthedocs-hosted.com/en/latest/frameworks/torch/torch-n...
2
answers
0
votes
452
views
asked a year ago
I am trying to run inference of GPT2 on inf2 instance using this transformers-neuronx example: [https://github.com/aws-neuron/transformers-neuronx#hugging-face-generate-api-support]() I keep getting ...
1
answers
0
votes
409
views
asked 2 years ago
I am currently facing an issue with the AWS Neuron SDK when trying to run the PyTorch example provided in the AWS Neuron GitHub repository on a Deep Learning AMI Neuron PyTorch 1.13 (Ubuntu 20.04) ins...
1
answers
0
votes
635
views
asked 2 years ago
Pytorch 2.0.1 is backward compatible so it shall be fine to use it but when we tried to we got this error. ImportError Invalid dependency version torch==2.0.1+cu117. Expected torch==1.13.1 ``` # ...
1
answers
0
votes
675
views
asked 2 years ago
The audio going into the model would be different lengths, and I saw a post saying "At the time of writing, the AWS Neuron SDK does not support dynamic shapes, which means that the input size needs to...
1
answers
0
votes
357
views
asked 2 years ago
Hi, I am trying to deploy the Databricks open source LLM i.e Dolly on inf2 instance. Instance type is `inf2.24xlarge` used the AMI `Deep Learning AMI Neuron PyTorch 1.13 (Ubuntu 20.04) 2023051`. I am ...
2
answers
0
votes
924
views
asked 2 years ago