Questions tagged with AWS Inferentia
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
It seems to be available according to every online source I see.
2
answers
0
votes
462
views
asked 10 months agolg...
I am currently facing an issue with the AWS Neuron SDK when trying to run the PyTorch example provided in the AWS Neuron GitHub repository on a Deep Learning AMI Neuron PyTorch 1.13 (Ubuntu 20.04)...
1
answers
0
votes
356
views
asked a year agolg...
I am currently using Amazon SageMaker for running my machine learning models, but it is becoming costly. To reduce costs, I am considering two options: AWS Elastic Inference and AWS Inferentia.
I...
1
answers
0
votes
777
views
asked a year agolg...
Hi All,
I have to compute gradient on BERT model on inferentia. For this I guess I also need access to the hidden layers. Im currently not able to proceed because of not finding literature on the net...
1
answers
0
votes
163
views
asked a year agolg...
I'm trying to make a public facing web app that allows for inferencing, with probably ten or so available models to my users. My initial thought was that I would have a front-end basic webpage, that...
1
answers
0
votes
222
views
asked a year agolg...
Hi,
I am trying to deploy the Databricks open source LLM i.e Dolly on inf2 instance. Instance type is `inf2.24xlarge` used the AMI `Deep Learning AMI Neuron PyTorch 1.13 (Ubuntu 20.04) 2023051`.
I am...
2
answers
0
votes
677
views
asked a year agolg...
Hi,
I have some code which generates a shape of torch.Size([1, 512, 1024] when calling bert on inf1.
I have compiled the model for inf2.
However the same code on inf2 produces a shape of...
1
answers
0
votes
278
views
asked a year agolg...
Hi, I'm trying to run the gptj_demo on Inf2 with AMI Deep Learning AMI Neuron PyTorch 1.13.0 (Ubuntu 20.04) 20230405 and installed the pytorch neuron as...
1
answers
0
votes
394
views
asked a year agolg...
I have an ML model from Huggingface, which essentially looks as follows:
```
import torch
from transformers import BloomTokenizerFast, BloomForCausalLM
device = torch.device('cuda' if...
0
answers
0
votes
87
views
asked a year agolg...
Dear developers,
I am relatively new to AWS and EC2 instances. I have an EC2 Inf1 instance and I am trying to set up tensorflow neuron for deep learning applications.
When I running the 'Resnet50...
2
answers
0
votes
324
views
asked a year agolg...
Diffusers aren't yet supported for deployment on Inf instances?
If they already are, what docs could be the guide to achieve it?
Beforehand, thank you.
1
answers
0
votes
253
views
asked a year agolg...
We have a huggingfacemodel with zero-shot-classification with neuron infernetia. It's based on [the pretrained huggingface pipelines distilBert with TensorFlow2...
1
answers
0
votes
306
views
asked a year agolg...