1 réponse
- Le plus récent
- Le plus de votes
- La plupart des commentaires
0
You have 2 options to consume a model via Bedrock, On-demand vs Provisioned Throughput.
As per the documentation https://docs.aws.amazon.com/bedrock/latest/userguide/quotas.html#quotas-runtime, latency differs by model and is directly proportional to the following conditions.
- The number of input and output tokens
- The total number of ongoing on-demand requests by all customers at the time.
You can purchase Provisioned Throughput to address your issue.
https://docs.aws.amazon.com/bedrock/latest/userguide/prov-throughput.html
Please have a look at the below thread for a similar issue.
https://repost.aws/questions/QUC82MTlWlQNagsqEG2Hbxlw/aws-bedrock-throttlingexception-occurs-randomly-for-claude-2-1-runtime
répondu il y a 2 mois
Contenus pertinents
- demandé il y a un an
- demandé il y a 2 mois
- demandé il y a un mois
- AWS OFFICIELA mis à jour il y a un an
- AWS OFFICIELA mis à jour il y a un an
- AWS OFFICIELA mis à jour il y a 4 mois
- AWS OFFICIELA mis à jour il y a un an