Questions tagged with AWS Inferentia
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Hello,
Difference between AWS Trainium, AWS inferentia and Instances with Habana Accelerators?
Thanks,
1
answers
0
votes
381
views
asked a year agolg...
I converted a pytorch BERT model to neuron. However the embedding or output tensors which is a list of 1024 size is different..i.e the list sizes are same but individual entries differ. Each of the...
2
answers
0
votes
495
views
asked a year agolg...
Hello,
We are testing the pipeline mode for neuron/inferentia, but can not get a model running for multi-core. The single core compiled model loads fine and is able to run inference on inferentia...
1
answers
0
votes
380
views
asked a year agolg...
Im using the following code to load a neuron compiled model for inference. However on my inf1.2xlarge instance, neuron-top shows for cores (NC0 to NC3). Only NC0 gets used in inference. How do I...
1
answers
0
votes
284
views
asked a year agolg...
Hi,
I want to neuron compile a bert large model(patentbert from google) which has sequence length 512. How do I do this?
Also I want to call the model as before or need to know what I should change...
1
answers
0
votes
397
views
asked a year agolg...
I am trying to load a neuron compiled model generated as given in https://awsdocs-neuron.readthedocs-hosted.com/en/latest/src/examples/tensorflow/huggingface_bert/huggingface_bert.html . I am still a...
2
answers
0
votes
361
views
asked a year agolg...
Hi,
This link https://awsdocs-neuron.readthedocs-hosted.com/en/latest/frameworks/tensorflow/tensorflow-neuron/tutorials/bert_demo/bert_demo.html mentions how to compile using tensorflow 1. Can anyone...
1
answers
0
votes
431
views
asked a year agolg...
I followed user guide on updating torch neuron and then started compiling the model to neuron.
But got an error, from which I don't understand what's wrong.
In Neuron SDK you claim that it should...
1
answers
1
votes
766
views
asked 2 years agolg...
I'm following some guides and from my understanding this should be possible. But I've been trying for hours to compile a yolov5 model into a neuron model with no success. Is it even possible to do...
1
answers
2
votes
655
views
asked 2 years agolg...
Hi Team,
I wanted to compile a BERT model and run it on inferentia. I trained my model using pytorch and tried to convert it by following the same steps in this...
1
answers
0
votes
321
views
asked 2 years agolg...
I am trying to test a model compiled for Inferentia on an `inf1.2xlarge`, but when loading the model I receive the following error messages:
```
2022-Sep-15 22:10:01.0152 3802:3802 ERROR ...
1
answers
0
votes
321
views
asked 2 years agolg...
I have compiled my model to run on Inferentia and I can load up multiple models from 1 process such as a single jupyter notebook.
I am trying to host the models via a server and am using gunicorn as...
2
answers
0
votes
459
views
asked 2 years agolg...