Questions tagged with AWS Neuron
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Hello, i'm using **pytorch-inference:2.0.1-gpu-py310-cu118-ubuntu20.04-sagemaker** AMI and following this guide...
1
answers
0
votes
496
views
asked 7 months agolg...
hello,
i'm using **pytorch-inference:2.0.1-gpu-py310-cu118-ubuntu20.04-sagemaker** image to run the service i'd like to switch to INF2 instance.
~~I think i can try to use...
1
answers
0
votes
551
views
asked 8 months agolg...
Hi,
One big difference I've noticed between the inf1 and inf2 development experience is that with inf2, we are not able to compile models outside of inf2 instances (unlike inf1 where we were able to...
2
answers
0
votes
274
views
asked 9 months agolg...
Which version of torch_neuronx is used in the example?
This is asked because neither the API Reference at [aws...
2
answers
0
votes
307
views
asked 9 months agolg...
I am trying to run inference of GPT2 on inf2 instance using this transformers-neuronx example:
[https://github.com/aws-neuron/transformers-neuronx#hugging-face-generate-api-support]()
I keep getting...
1
answers
0
votes
294
views
asked a year agolg...
I am currently facing an issue with the AWS Neuron SDK when trying to run the PyTorch example provided in the AWS Neuron GitHub repository on a Deep Learning AMI Neuron PyTorch 1.13 (Ubuntu 20.04)...
1
answers
0
votes
502
views
asked a year agolg...
Pytorch 2.0.1 is backward compatible so it shall be fine to use it but when we tried to we got this error.
ImportError
Invalid dependency version torch==2.0.1+cu117. Expected torch==1.13.1
```
#...
1
answers
0
votes
480
views
asked a year agolg...
The audio going into the model would be different lengths, and I saw a post saying "At the time of writing, the AWS Neuron SDK does not support dynamic shapes, which means that the input size needs to...
1
answers
0
votes
263
views
asked a year agolg...
Hi,
I am trying to deploy the Databricks open source LLM i.e Dolly on inf2 instance. Instance type is `inf2.24xlarge` used the AMI `Deep Learning AMI Neuron PyTorch 1.13 (Ubuntu 20.04) 2023051`.
I am...
2
answers
0
votes
745
views
asked a year agolg...
Hi,
I have some code which generates a shape of torch.Size([1, 512, 1024] when calling bert on inf1.
I have compiled the model for inf2.
However the same code on inf2 produces a shape of...
1
answers
0
votes
321
views
asked a year agolg...
Hi, I'm trying to run the gptj_demo on Inf2 with AMI Deep Learning AMI Neuron PyTorch 1.13.0 (Ubuntu 20.04) 20230405 and installed the pytorch neuron as...
1
answers
0
votes
433
views
asked a year agolg...
I am trying to compile a Roberta-large model for inf1 instance for batch size 4. I was successfully able to compile the model for batch size = 1, but getting the following error when trying to compile...
3
answers
0
votes
403
views
asked a year agolg...