AWS bedrock Claude v2 API - go sdk v2

Question

I am programatically calling claude v2 api using Go sdk 
```
import 	"github.com/aws/aws-sdk-go-v2/service/bedrockruntime"

output, err := app.brc.InvokeModel(context.Background(), &bedrockruntime.InvokeModelInput{
		Body:        payloadBytes,
		ModelId:     aws.String("anthropic.claude-v2"),
		ContentType: aws.String("application/json"),
	})
```
___
Most of the times, it takes more than 5 seconds which is significantly slower than the responsiveness of bedrock playground. Is anything wrong with the code, or any suggestions for improving response time?

Answer

Hi,

As you say "v2", did you try v2.1 ? See https://aws.amazon.com/about-aws/whats-new/2023/11/claude-2-1-foundation-model-anthropic-amazon-bedrock/

The announcement above doesn't much resp time enhancement. But, as it is the new favored model, it may receive more computing resources than v2.

Best,

Didier

AWS bedrock Claude v2 API - go sdk v2

Contenuto pertinente