Issue importing magistral in AWS bedrock, cannot find model weight file
Hello,
I am trying to use Mistral Magistral small with bedrock, as it's not part of the marketplace i am trying to import it. Let me recap what i have done so far :
First step i did was to download the content of this site https://huggingface.co/mistralai/Magistral-Small-2506/tree/main into the root of an s3 bucket, i took the folowing files : -tekken.jon -params.json -config.json -consolidated.safetensors
then try to import it so i get an error of "Amazon bedrock could not find the expected file tokenizer_config.json " So i found this file here https://huggingface.co/mlx-community/Magistral-Small-2506-bf16/blob/main/tokenizer_config.json and add it to my s3 bucket.
I tried again to import and got next error "Amazon Bedrock could not load the tokenizer. Make sure you have included all necessary tokenizer files in your S3 location, and try again. Example tokenizer files are: tokenizer.json, special_tokens_map.json, added_tokens.json, tokenizer.model." So I took the file here https://huggingface.co/mlx-community/Magistral-Small-2506-4bit-DWQ/blob/main/tokenizer.json and upload it to the root of my s3 bucket.
Now i am getting the error with the missing weight and cannot find any solutions. Why is bedrock not finding my consolidated.safetensors file ?
This is what my S3 bucket look like :
Is there a specific way i need to organize my s3 bucket ?
Cheers
Vincent
- Newest
- Most votes
- Most comments
When importing a custom model into Amazon Bedrock, you're encountering an issue with the model weights file. There are several requirements and potential solutions to address this problem:
First, for a successful model import into Amazon Bedrock, you need specific files in the Hugging Face weights format:
- .safetensors file (model weights in Safetensor format)
- config.json
- tokenizer_config.json
- tokenizer.json
- tokenizer.model
Looking at your S3 bucket screenshot, I notice you have config.json, consolidated.safetensors, params.json, tekken.jon, and tokenizer.json, but you're missing the tokenizer.model file which is required for the import process.
Additionally, there could be issues with the safetensor file itself. Amazon Bedrock only supports model weights with precision formats FP32, FP16, and BF16. If your consolidated.safetensors file is in a different format (like a quantized format), it won't be recognized properly.
For Mistral models specifically, you might need to convert the weights to the proper format. You can use the Mistral convert scripts to convert your model to a supported format if needed.
To resolve this issue:
- Make sure you have all required files, including tokenizer.model
- Verify that your safetensor file is valid and in a supported precision format
- Ensure all files are at the root level of your S3 bucket (which appears to be the case from your screenshot)
- If you're still having issues, you might need to download the model files from the official Mistral repository rather than from different community repositories, as mixing files from different sources could cause compatibility issues
If the error persists, you might need to check if the consolidated.safetensors file is corrupted or if it's in an incompatible format for Amazon Bedrock.
Sources
Import a customized model into Amazon Bedrock - Amazon Bedrock
Amazon Bedrock Custom Model Import now generally available | AWS Machine Learning Blog
The Safetensor files that you provided are not valid. Make sure the model weights are valid and that you can load them with the Safetensor method | AWS re:Post
Relevant content
- asked 8 months ago
Hello, I add the file tokenizer.model from this url https://huggingface.co/mistral-community/Mixtral-8x22B-v0.1-AWQ/blob/5992b45ecd439cf09c2e26724a0ff4a71c81619d/tokenizer.model but the error message ("Amazon Bedrock could not find the model weight files. Make sure model weight are in a supported format and try again. The supported formats are: [HuggingFace_SafeTensor].") remain the same.
Vincent