tldr: InvokeAgentCommand returns data inside trace but the data chunks themselves are missing
hello AWS people,
I have a weird issue at hand. (setup and config will follow below)
inside a Lambda function and using the Bedrock Agent Runtime, when we invoke the InvokeAgentCommand (with streaming enabled and tracing enabled) we do get back a 200 OK response.
HOWEVER, the data chunks are missing completely. here comes the weird part - inside the last trace object, we see the final response which otherwise would have been part of the data chunks...
how? why?
by inspecting the traces we see that the agent otherwise works well - it understands what information it has to get, goes to the knowledge base, extracts some data, then narrows it down to what it needs, finally it prepares a final response which is correct, indeed.
we tried both dev (TSTALIASID/DRAFT) and prod (agent alias ID/version ID) alternatives of the agent but the result is always the same.
we asked dear Claude for help himself - no success.
we read 3 similar questions here in re:Post - no success.
we kindly ask the AWS experts and the community to give us a hand.
--- details ---
region: eu-central-1
model name: Claude 3.5 Sonnet
inference profile: eu.anthropic.claude-3-5-sonnet-20240620-v1:0
agent alias ID: TSTALIASID
agent version: Draft
lambda runtime: nodejs22.x
bedrock agent sdk and version: "@aws-sdk/client-bedrock-agent-runtime": "^3.717.0"
--- infra (only the relevant bits) ---
resource "aws_bedrock_inference_profile" "sonnet" {
name = "goajent-${var.environment}-sonnet-inference-profile"
model_source {
copy_from = "arn:aws:bedrock:eu-central-1:${local.account_id}:inference-profile/eu.anthropic.claude-3-5-sonnet-20240620-v1:0"
}
}
resource "aws_bedrockagent_agent" "fff" {
agent_name = "fff-${var.environment}-agent"
agent_resource_role_arn = aws_iam_role.bedrock_agent.arn
foundation_model = aws_bedrock_inference_profile.sonnet.arn
instruction = file("${path.module}/agent-instructions/instructions.txt")
}
--- code (only the relevant bits) ---
const input = {
agentId: BEDROCK_AGENT_ID,
agentAliasId: BEDROCK_AGENT_VERSION, // it is "TSTALIASID"
sessionId,
endSession,
enableTrace: true,
inputText: prompt,
streamingConfigurations: {
streamFinalResponse: true,
},
}
const command = new InvokeAgentCommand(input)
const response = await client.send(command)
for await (const chunkEvent of response.completion) {
if (chunkEvent.chunk) {
console.log("DATA: \n", JSON.stringify(chunkEvent.chunk))
} else if (chunkEvent.trace) {
console.info("TRACE: \n", JSON.stringify(chunkEvent.trace))
}
}
(again - we never see the DATA log but always do see the TRACE log.)
--- an example of the final trace object (redacted) ---
{
"agentAliasId": "TSTALIASID",
"agentId": "fff",
"agentVersion": "DRAFT",
"callerChain": [{ "agentAliasArn": "arn:aws:bedrock:eu-central-1:fff:agent-alias/fff/TSTALIASID" }],
"sessionId": "fff",
"trace": {
"orchestrationTrace": {
"observation": {
"finalResponse": {
"text": "Based on the provided transaction data, I can summarize two ff transactions for you:\n\n1. On ff, there was an ff payment of €ff.00 to ff, likely for a service or product from ff.\n\n2. ff, there was an ff payment of €ff.00 to ff\n\nThese are the only ff transactions visible in the given transaction history."
},
"traceId": "ff",
"type": "FINISH"
}
}
}
}
okay, this gets more weird by the minute. based on your suggestion, dear agent, I turned off the tracing. I invoked the function 3 times. only one time it did return a data chunk. but it did return a single chunk with the final response in it. technically, it was streamed but in a single chunk only....
it would be nice if a human could jump on this. I think it's an interesting problem. this bevaviour of sometimes returning nothing to sometimes returning a single chunk versus streaming multiple chunks on every invocations seems really not expected and very strange. somebody from the Bedrock team jumping on this would be great. or anybody that has experience building agents with the Agent Runtime.