- Newest
- Most votes
- Most comments
Hello,
The start-async-invoke API is for video generation with Amazon Nova Reel: User Guide for Amazon Nova - Video generation access and usage.
You mentioned that you want to invoke Amazon Nova Lite or Amazon Nova Pro. Could you please provide some more details what you are trying to achieve. Do you want to invoke Amazon Nova Lite or Amazon Nova Pro with a batch of prompts (batch inference)?
With batch inference you can submit multiple prompts and generate responses asynchronously. Amazon Nova Lite and Amazon Nova Pro support batch inference: Amazon Bedrock User Guide - Supported Regions and models for batch inference.
Here is a code sample that shows how to run batch inference with Amazon Nova Foundation Models: GitHub Amazon Nova Samples - Batch Inference Text.
As of the current information available, it appears that async inference support in Amazon Bedrock is limited and not widely available across models. While the StartAsyncInvoke API exists, there seems to be a lack of clear documentation or examples showing which specific models support this feature.
You mentioned trying Nova Lite and Pro models, but receiving the error "The provided model doesn't support async inference." This suggests that even some of the more advanced models in Bedrock do not yet support async inference.
The Amazon Nova models, including Nova Micro, Nova Lite, and Nova Pro, are known to support various features and APIs within Bedrock, such as the Invoke API (InvokeModel, InvokeModelWithResponseStream) and the Converse API (Converse and ConverseStream). However, explicit support for async inference is not clearly stated for these models.
Given the lack of clear documentation on which models support async inference and the error messages you've encountered, it seems that this feature might be in a limited release or still under development for most Bedrock models.
As you noted, using AWS Step Functions (Flow) could be a viable alternative to build out asynchronous processing capabilities if direct async inference support is not available for your desired models.
If you need to use async inference specifically, it would be advisable to contact AWS support directly for the most up-to-date information on which models, if any, currently support this feature in Bedrock. They may be able to provide you with more specific guidance or information about upcoming support for async inference across different models.
Sources
Amazon Nova models - Amazon Bedrock
AsyncInvokeSummary - Amazon Bedrock
Thanks, AI, for pointing out the lack of clear documentation! I agree with you 100%.
Relevant content
- asked 5 months ago
- asked 5 months ago
AWS OFFICIALUpdated 9 days ago
Thanks for the answer!
No, I am not doing any batch inference. I use the Converse API to make single invocations and I was hoping to use start-async-invoke to make similar invocations but done asynchronously. Evidently most models are not supported with start-async-invoke so I will stay away from that until I use Reel. Perhaps I will try Flow.
You are welcome! What programming language are you using?
We use the Ruby SDK running on Lambdas. I was interested to see if start-async-invoke could be used to simplify our workflow. If it supported more models it could be useful.
Thank you for providing the additional details. I'll take the feedback to add more models to the start-async-invoke API.
In the meantime you could use threading or sub-processes to run other code while you are making an InvokeModel or Converse API request. Here are more details: https://github.com/aws/aws-sdk-ruby/issues/354#issuecomment-23998769