AWS bedrock Claude v2 API - go sdk v2

I am programatically calling claude v2 api using Go sdk

import 	"github.com/aws/aws-sdk-go-v2/service/bedrockruntime"

	output, err := app.brc.InvokeModel(context.Background(), &bedrockruntime.InvokeModelInput{
		Body:        payloadBytes,
		ModelId:     aws.String("anthropic.claude-v2"),
		ContentType: aws.String("application/json"),
	})

Most of the times, it takes more than 5 seconds which is significantly slower than the responsiveness of bedrock playground. Is anything wrong with the code, or any suggestions for improving response time?

주제

기계 학습 및 AI AWS의 제너레이티브 AI

태그

아마존 Bedrock

언어

English

Arvi

질문됨 3달 전254회 조회

1개 답변

최신
최다 투표
가장 많은 댓글

이 답변이 도움이 되었나요?커뮤니티가 여러분의 지식을 활용할 수 있도록 정답을 찬성하세요.

Hi,

As you say "v2", did you try v2.1 ? See https://aws.amazon.com/about-aws/whats-new/2023/11/claude-2-1-foundation-model-anthropic-amazon-bedrock/

The announcement above doesn't much resp time enhancement. But, as it is the new favored model, it may receive more computing resources than v2.

Best,

Didier

전문가

Didier_Durand

답변함 3달 전

Arvi

3달 전

Hey, thanks for the response. I tried v2.1 but issue still persists. updated code looks like this

	output, err := app.brc.InvokeModel(context.Background(), &bedrockruntime.InvokeModelInput{
		Body:        payloadBytes,
		ModelId:     aws.String("anthropic.claude-v2:1"),
		ContentType: aws.String("application/json"),
	})

region : us-east-1

AWS bedrock Claude v2 API - go sdk v2

관련 콘텐츠