Update Text Embedding Inference container image

Question

Huggingface (https://github.com/huggingface/text-embeddings-inference) has released the image to support the new embedding models of Qwen. I really need them for our project. But the latest tei-huggingface image provided by AWS is just 1.7, which does not support the models.

Could you please add this support?

Besides, I couldn't find the information on the available of the tei images anywhere on the AWS Deep Learning Container repository.

Kidd Ip · Answer

This can be a bug topic, in short, you may start to deploy the latest Qwen model manually:

```
model=Qwen/Qwen3-Embedding-0.6B
volume=$PWD/data

docker run --gpus all -p 8080:80 -v $volume:/data --pull always \
ghcr.io/huggingface/text-embeddings-inference:1.7 \
--model-id $model
```

This pulls the latest image from Hugging Face’s GitHub Container Registry and loads the Qwen model directly.

Taz · Answer

Hey,

Hope you're keeping well.

AWS Deep Learning Containers are versioned and published on the [AWS Deep Learning Container images page](https://docs.aws.amazon.com/deep-learning-containers/latest/devguide/deep-learning-containers-images.html), but they may lag behind upstream releases from Hugging Face. If you need Qwen embedding support immediately, you can build a custom SageMaker inference container by starting from the Hugging Face `text-embeddings-inference` image on GHCR and adding your required model, then push it to Amazon ECR for deployment. This approach lets you run the latest TEI version in SageMaker or ECS without waiting for the official AWS DLC update.

Thanks and regards,  
Taz

Update Text Embedding Inference container image

Add your answer

Relevant content