AWS bedrock Claude v2 API - go sdk v2

1

I am programatically calling claude v2 api using Go sdk

import 	"github.com/aws/aws-sdk-go-v2/service/bedrockruntime"

	output, err := app.brc.InvokeModel(context.Background(), &bedrockruntime.InvokeModelInput{
		Body:        payloadBytes,
		ModelId:     aws.String("anthropic.claude-v2"),
		ContentType: aws.String("application/json"),
	})

Most of the times, it takes more than 5 seconds which is significantly slower than the responsiveness of bedrock playground. Is anything wrong with the code, or any suggestions for improving response time?

Arvi
asked 3 months ago265 views
1 Answer
0

Hi,

As you say "v2", did you try v2.1 ? See https://aws.amazon.com/about-aws/whats-new/2023/11/claude-2-1-foundation-model-anthropic-amazon-bedrock/

The announcement above doesn't much resp time enhancement. But, as it is the new favored model, it may receive more computing resources than v2.

Best,

Didier

profile pictureAWS
EXPERT
answered 3 months ago
  • Hey, thanks for the response. I tried v2.1 but issue still persists. updated code looks like this

    	output, err := app.brc.InvokeModel(context.Background(), &bedrockruntime.InvokeModelInput{
    		Body:        payloadBytes,
    		ModelId:     aws.String("anthropic.claude-v2:1"),
    		ContentType: aws.String("application/json"),
    	})
    

    region : us-east-1

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions