Questions tagged with AWS Inferentia
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
It seems to be available according to every online source I see.
2
answers
0
votes
447
views
asked 10 months agolg...
I am currently facing an issue with the AWS Neuron SDK when trying to run the PyTorch example provided in the AWS Neuron GitHub repository on a Deep Learning AMI Neuron PyTorch 1.13 (Ubuntu 20.04)...
1
answers
0
votes
341
views
asked 10 months agolg...
I am currently using Amazon SageMaker for running my machine learning models, but it is becoming costly. To reduce costs, I am considering two options: AWS Elastic Inference and AWS Inferentia.
I...
1
answers
0
votes
748
views
asked 10 months agolg...
Hi All,
I have to compute gradient on BERT model on inferentia. For this I guess I also need access to the hidden layers. Im currently not able to proceed because of not finding literature on the net...
1
answers
0
votes
159
views
asked a year agolg...
I'm trying to make a public facing web app that allows for inferencing, with probably ten or so available models to my users. My initial thought was that I would have a front-end basic webpage, that...
1
answers
0
votes
207
views
asked a year agolg...
Hi,
I am trying to deploy the Databricks open source LLM i.e Dolly on inf2 instance. Instance type is `inf2.24xlarge` used the AMI `Deep Learning AMI Neuron PyTorch 1.13 (Ubuntu 20.04) 2023051`.
I am...
2
answers
0
votes
645
views
asked a year agolg...
Hi,
I have some code which generates a shape of torch.Size([1, 512, 1024] when calling bert on inf1.
I have compiled the model for inf2.
However the same code on inf2 produces a shape of...
1
answers
0
votes
263
views
asked a year agolg...
Hi, I'm trying to run the gptj_demo on Inf2 with AMI Deep Learning AMI Neuron PyTorch 1.13.0 (Ubuntu 20.04) 20230405 and installed the pytorch neuron as...
1
answers
0
votes
379
views
asked a year agolg...
I have an ML model from Huggingface, which essentially looks as follows:
```
import torch
from transformers import BloomTokenizerFast, BloomForCausalLM
device = torch.device('cuda' if...
0
answers
0
votes
81
views
asked a year agolg...
Dear developers,
I am relatively new to AWS and EC2 instances. I have an EC2 Inf1 instance and I am trying to set up tensorflow neuron for deep learning applications.
When I running the 'Resnet50...
2
answers
0
votes
310
views
asked a year agolg...
Diffusers aren't yet supported for deployment on Inf instances?
If they already are, what docs could be the guide to achieve it?
Beforehand, thank you.
1
answers
0
votes
243
views
asked a year agolg...
We have a huggingfacemodel with zero-shot-classification with neuron infernetia. It's based on [the pretrained huggingface pipelines distilBert with TensorFlow2...
1
answers
0
votes
300
views
asked a year agolg...