Skip to content

Using tools within Bedrock Flows

0

Hi there!

I have a multi prompt flow set up and it works fine. I successfully get all outputs streamed to me as part of the BedrockAgentRuntimeClient responseStream However, if I add tool use to the prompt in the prompt manager, validated to work when testing the prompt the "document" is returned as an empty string.

I cant find any documentation on this. Configuration of neither prompt output nor outputNode input seem like they are still prototypes.

Am I missing something or is this just not yet supported?

Tool Spec :

{
    "toolChoice": {
        "tool": {
            "name": "apply_tags"
        }
    },
    "tools": [
        {
            "toolSpec": {
                "description": "Apply tags",
                "inputSchema": {
                    "json": {
                        "type": "object",
                        "properties": {
                            "tags": {
                                "type": "array",
                                "description": "An array of tags to apply",
                                "items": {
                                    "type": "string"
                                }
                            }
                        },
                        "required": [
                            "tags"
                        ]
                    }
                },
                "name": "apply_tags"
            }
        }
    ]
}

Flow overview

Enter image description here

Test calling code

import {
  BedrockAgentRuntimeClient,
  InvokeFlowCommand,
} from "@aws-sdk/client-bedrock-agent-runtime";

/**
 * Invokes an alias of a flow to run the inputs that you specify and return
 * the output of each node as a stream.
 *
 * @param {{
 *  flowIdentifier: string,
 *  flowAliasIdentifier: string,
 *  prompt?: string,
 *  region?: string
 * }} options
 * @returns {Promise<Array>} An array containing all flow output and completion events.
 */
export const invokeBedrockFlow = async ({
  flowIdentifier,
  flowAliasIdentifier,
  document,
  region = "us-east-2",
}) => {
  const client = new BedrockAgentRuntimeClient({ region });

  const command = new InvokeFlowCommand({
    flowIdentifier,
    flowAliasIdentifier,
    inputs: [
      {
        content: {
          document,
        },
        nodeName: "FlowInputNode",
        nodeOutputName: "document",
      },
    ],
    enableTrace: true,
  });

  const flowEvents = [];
  const response = await client.send(command);

  for await (const chunkEvent of response.responseStream) {
    const eventToProcess = chunkEvent.flowOutputEvent || chunkEvent.flowCompletionEvent

    if (eventToProcess) {
      flowEvents.push(Object.fromEntries(Object.entries(eventToProcess)))
    }
  }


  return flowEvents;
};

const response = await invokeBedrockFlow({
  flowIdentifier: "blablabla",
  flowAliasIdentifier: "blablabla",
  document: {
    prompt: 'roses are red, violets are blue',
    metadata: {
      userName: 'John Doe'
    }
  },
  region: "us-east-2",
});

console.log(JSON.stringify(response, null, 2));

Response without tool

[
  {
    "content": {
      "document": "\"tag,roses,red,violets,blue,flowers,poetry,rhyme,classic,verse,romantic,garden,blooms,petals,color,nature,love,traditional,simple,nursery,couplet\""
    },
    "nodeName": "FlowOutputNode"
  },
  {
    "completionReason": "SUCCESS"
  }
]
# Response with tool 
```json
[
  {
    "content": {
      "document": ""
    },
    "nodeName": "FlowOutputNode"
  },
  {
    "completionReason": "SUCCESS"
  }
]
2 Answers
1

Based on the information you've provided, it appears that you're encountering an issue with tool use in Amazon Bedrock Flows. While tool use is supported in Amazon Bedrock's Converse API, there are some limitations and considerations when it comes to using tools within Bedrock Flows.

Currently, the integration of tool use within Bedrock Flows is not fully supported or documented in the same way as it is for the Converse API. The behavior you're experiencing, where the "document" is returned as an empty string when adding tool use to the prompt, suggests that the flow is not properly handling or processing the tool use request.

This limitation is likely due to the current implementation of Bedrock Flows, which may not have full support for the tool use functionality that's available in the Converse API. Flows are designed to process inputs and outputs in a specific way, and the tool use feature may not be fully integrated into this workflow yet.

To work around this limitation, you have a few options:

  1. Use the Converse API directly: If tool use is critical for your use case, you might consider using the Amazon Bedrock Converse API directly instead of Flows. This would allow you to implement tool use as demonstrated in other examples.

  2. Break down the task: You could potentially break your flow into multiple steps, handling the tool use outside of the flow and then feeding the results back into the flow as a separate input.

  3. Wait for updates: As Amazon Bedrock and its features are continuously evolving, it's possible that full support for tool use within Flows may be added in the future. Keep an eye on AWS documentation and announcements for updates.

  4. Provide feedback: Consider reaching out to AWS support or providing feedback about this limitation. AWS often takes user feedback into account when prioritizing feature development.

In the meantime, if you need to use tools within your AI workflow, you might need to implement this functionality outside of Bedrock Flows, possibly by using the Converse API directly or by handling tool use in your application logic before or after invoking the flow.
Sources
Multi-turn interaction is not supported for observation type: ASK_USER | AWS re:Post
Community | Intro to Tool Use with the Amazon Bedrock Converse API
Unable to use tools with Bedrock Runtime: Converse Step Function Task | AWS re:Post

answered a year ago

EXPERT

reviewed a year ago

0

The AI response here is pure junk. None of the sources mention tool use in flows, the response is at best inference at worst pure hallucination.

The lack of documented feature support here is worrying. It's quite frustrating that the lack of certain capabilities is very hidden. Did I miss a supported features table or some such somewhere?

answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.