Questions tagged with AWS Inferentia
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Hello,
Difference between AWS Trainium, AWS inferentia and Instances with Habana Accelerators?
Thanks,
1
answers
0
votes
390
views
asked a year agolg...
I converted a pytorch BERT model to neuron. However the embedding or output tensors which is a list of 1024 size is different..i.e the list sizes are same but individual entries differ. Each of the...
2
answers
0
votes
507
views
asked a year agolg...
Hello,
We are testing the pipeline mode for neuron/inferentia, but can not get a model running for multi-core. The single core compiled model loads fine and is able to run inference on inferentia...
1
answers
0
votes
394
views
asked a year agolg...
Im using the following code to load a neuron compiled model for inference. However on my inf1.2xlarge instance, neuron-top shows for cores (NC0 to NC3). Only NC0 gets used in inference. How do I...
1
answers
0
votes
292
views
asked a year agolg...
Hi,
I want to neuron compile a bert large model(patentbert from google) which has sequence length 512. How do I do this?
Also I want to call the model as before or need to know what I should change...
1
answers
0
votes
407
views
asked 2 years agolg...
I am trying to load a neuron compiled model generated as given in https://awsdocs-neuron.readthedocs-hosted.com/en/latest/src/examples/tensorflow/huggingface_bert/huggingface_bert.html . I am still a...
2
answers
0
votes
373
views
asked 2 years agolg...
Hi,
This link https://awsdocs-neuron.readthedocs-hosted.com/en/latest/frameworks/tensorflow/tensorflow-neuron/tutorials/bert_demo/bert_demo.html mentions how to compile using tensorflow 1. Can anyone...
1
answers
0
votes
443
views
asked 2 years agolg...
I followed user guide on updating torch neuron and then started compiling the model to neuron.
But got an error, from which I don't understand what's wrong.
In Neuron SDK you claim that it should...
1
answers
1
votes
776
views
asked 2 years agolg...
I'm following some guides and from my understanding this should be possible. But I've been trying for hours to compile a yolov5 model into a neuron model with no success. Is it even possible to do...
1
answers
2
votes
669
views
asked 2 years agolg...
Hi Team,
I wanted to compile a BERT model and run it on inferentia. I trained my model using pytorch and tried to convert it by following the same steps in this...
1
answers
0
votes
333
views
asked 2 years agolg...
I am trying to test a model compiled for Inferentia on an `inf1.2xlarge`, but when loading the model I receive the following error messages:
```
2022-Sep-15 22:10:01.0152 3802:3802 ERROR ...
1
answers
0
votes
329
views
asked 2 years agolg...
I have compiled my model to run on Inferentia and I can load up multiple models from 1 process such as a single jupyter notebook.
I am trying to host the models via a server and am using gunicorn as...
2
answers
0
votes
469
views
asked 2 years agolg...