AWS Bedrock Batch Inference Operation Not Allowed

Question

I'm trying to perform batch inference on the Meta Llama 3.2 90B model from the us-east-1 region (via cross-region inference).

So far, I've been able to set up an S3 bucket with my input file (1000 requests) and output directory. 
I've tried running a batch inference job with AWS Bedrock console and automatically tried creating a service role. However, I get two errors saying Operation not allowed.

I tried making my own role (including one with full administrator access) and validating with the IAM Policy Simulator. I also ran an example code from the documentation: 
https://docs.aws.amazon.com/bedrock/latest/userguide/batch-inference-example.html

However, the only error I get is `Operation not allowed.` Inspecting the debugging logs, the error occurred with the POST "/model-invocation-job" request. I have double-checked the input file to make sure that the inputs are valid.

I also checked the cloud trail logs, but nothing was outputted there.

Is there a workaround/fix? Or am I only able to make individual requests but not batch inference?

Riku_Kobayashi · Answer

Hello.

It's possible that your AWS account has restrictions on using Bedrock.  
In this case, I recommend that you open a case with AWS Support under "Account and billing" since there may not be anything you can do to resolve the issue.  
https://repost.aws/ja/questions/QUHAyQFm2LTqOhLibr5Z9K4Q/aws-bedrock-model-access-results-in-operation-not-allowed?sc_ichannel=ha&sc_ilang=en&sc_isite=repost&sc_iplace=hp&sc_icontent=QUHAyQFm2LTqOhLibr5Z9K4Q&sc_ipos=16  
https://stackoverflow.com/questions/78135644/aws-bedrock-not-able-to-get-access-error-operation-not-allowed

re:Post Agent · Answer

The issue you're experiencing with AWS Bedrock Batch Inference for the Meta Llama 3.2 90B model in the us-east-1 region appears to be related to model availability and service limitations rather than a configuration problem on your end.

Batch inference capabilities can vary by model and region, and not all models may support batch inference in all regions. The "Operation not allowed" error you're encountering suggests that there might be restrictions on batch inference for this specific model or in the region you're trying to use.

To troubleshoot and potentially resolve this issue:

1. Verify model availability: Check the AWS Bedrock documentation to confirm that the Meta Llama 3.2 90B model supports batch inference in the us-east-1 region. Model availability can change over time and vary by region.

2. Try a different region: If possible, attempt to run the batch inference job in a different region where the model is known to support batch inference.

3. Test with a different model: Try creating a batch inference job with a different model (e.g., one of the Claude models) to see if the issue is specific to the Meta Llama 3.2 90B model.

4. Individual requests: If batch inference is not supported for this model, you may need to fall back to making individual requests instead of using batch inference.

5. Contact AWS Support: If you've confirmed that batch inference should be supported for this model and region, and you've exhausted other troubleshooting steps, it would be advisable to open a support case with AWS. They can provide more specific guidance based on your account and usage details, and can also check if there are any known issues or limitations affecting your use case.

Remember that AWS services and capabilities are continually evolving, so it's possible that batch inference support for this model may be added or changed in the future if it's not currently available.  
**Sources**  
[Error in AWS Bedrock Batch Inteference | AWS re:Post](https://repost.aws/questions/QUBN3gF6eNSqSpniExiexuMQ/error-in-aws-bedrock-batch-inteference)  
[Batch inference general operation error | AWS re:Post](https://repost.aws/questions/QUamrH1jdJSNGk_cPyjJ0r6A/batch-inference-general-operation-error)  
[Bedrock Batch Inference with Claude | AWS re:Post](https://repost.aws/questions/QUNJdWhyqNQS6nQ6FUPPgC7w/bedrock-batch-inference-with-claude)  
[Amazon Bedrock batch job: Validation Error: Operation Not Allowed | AWS re:Post](https://repost.aws/questions/QUFB1qX-TLQt-ho2jQAvOYAQ/amazon-bedrock-batch-job-validation-error-operation-not-allowed)

AWS Bedrock Batch Inference Operation Not Allowed

Add your answer

Relevant content