Skip to content

converse api function calling empty response

0

Im having issues with function calling when i request more than 1 item from a dummy api. Here is the code

tool_list = [
    {
        "toolSpec": {
            "name": "quotes",
            "description": "Fetch a random quote.",
            "inputSchema": {
                "json": {
                    "type": "object",
                    "properties": {
                        "limit": {
                            "type": "number",
                            "description": "The number of quotes we want to fetch.",
                        }
                    },
                    "required": ["limit"],
                }
            },
        }
    }
]

message_list = []

initial_message = {
    "role": "user",
    "content": [{"text": "Can you get me a random quote?"}],
}

message_list.append(initial_message)

response = bedrock.converse(
    modelId="anthropic.claude-3-sonnet-20240229-v1:0",
    messages=message_list,
    inferenceConfig={"maxTokens": 2000, "temperature": 0},
    toolConfig={"tools": tool_list},
    system=[{"text": "You can only provide quotes by using a tool."}],
)

response_message = response["output"]["message"]
print(json.dumps(response_message, indent=4))
message_list.append(response_message)

response_content_blocks = response_message["content"]

follow_up_content_blocks = []

for content_block in response_content_blocks:
    if "toolUse" in content_block:
        tool_use_block = content_block["toolUse"]
        tool_use_name = tool_use_block["name"]

        if tool_use_name == "quotes":
            url = (
                f"http://dummyjson.com/quotes/random/{tool_use_block['input']['limit']}"
            )
            tool_result_value = requests.get(url)

            follow_up_content_blocks.append(
                {
                    "toolResult": {
                        "toolUseId": tool_use_block["toolUseId"],
                        "content": [{"json": {"result": tool_result_value.json()}}],
                    }
                }
            )

if len(follow_up_content_blocks) > 0:

    follow_up_message = {
        "role": "user",
        "content": follow_up_content_blocks,
    }

    message_list.append(follow_up_message)

    response = bedrock.converse(
        modelId="anthropic.claude-3-sonnet-20240229-v1:0",
        messages=message_list,
        inferenceConfig={"maxTokens": 2000, "temperature": 0},
        toolConfig={"tools": tool_list},
        system=[{"text": "You can only provide quotes by using a tool."}],
    )
    response_message = response["output"]["message"]
    print(json.dumps(response_message, indent=4))
    message_list.append(response_message)

If i run that code this is the output and everything works.

{
    "role": "assistant",
    "content": [
        {
            "text": "Okay, let's fetch a random quote:"
        },
        {
            "toolUse": {
                "toolUseId": "tooluse_mmss3FiWTlKShgIgMafVkw",
                "name": "quotes",
                "input": {
                    "limit": 1
                }
            }
        }
    ]
}
#### Response from dummy api
[{'id': 843, 'quote': 'A Nation That Continues Year After Year To Spend More Money On Military Defense Than On Programs Of Social Uplift Is Approaching Spiritual Doom.', 'author': 'Martin Luther King  Jr.'}]
#### My complete messages
[{'role': 'user', 'content': [{'text': 'Can you get me a quote?'}]}, {'role': 'assistant', 'content': [{'text': "Okay, let's fetch a random quote:"}, {'toolUse': {'toolUseId': 'tooluse_mmss3FiWTlKShgIgMafVkw', 'name': 'quotes', 'input': {'limit': 1}}}]}, {'role': 'user', 'content': [{'toolResult': {'toolUseId': 'tooluse_mmss3FiWTlKShgIgMafVkw', 'content': [{'json': {'result': [{'id': 843, 'quote': 'A Nation That Continues Year After Year To Spend More Money On Military Defense Than On Programs Of Social Uplift Is Approaching Spiritual Doom.', 'author': 'Martin Luther King  Jr.'}]}}]}}]}]
####Final response from bedrock
{
    "role": "assistant",
    "content": [
        {
            "text": "Here is a random quote for you:\n\n\"A Nation That Continues Year After Year To Spend More Money On Military Defense Than On Programs Of Social Uplift Is Approaching Spiritual Doom.\"\n- Martin Luther King Jr."
        }
    ]
}

If i instead change my initial prompt to request more than one random quote this is my output

{
    "role": "assistant",
    "content": [
        {
            "text": "Here is how we can get 2 random quotes:"
        },
        {
            "toolUse": {
                "toolUseId": "tooluse_kuk3BxTXRgCW-WiQQriDAw",
                "name": "quotes",
                "input": {
                    "limit": 2
                }
            }
        }
    ]
}
#### Response from dummy api
[{'id': 424, 'quote': "I hated every minute of training, but I said, “Don't quit. Suffer now and live the rest of your life as a champion.”", 'author': 'Muhammad Ali'}, {'id': 1052, 'quote': 'How Hard, How Bitter It Is To Become A Man!', 'author': 'Albert Camus'}]
#### Complete list of messages
[{'role': 'user', 'content': [{'text': 'Can you get me 2 random quotes?'}]}, {'role': 'assistant', 'content': [{'text': 'Here is how we can get 2 random quotes:'}, {'toolUse': {'toolUseId': 'tooluse_kuk3BxTXRgCW-WiQQriDAw', 'name': 'quotes', 'input': {'limit': 2}}}]}, {'role': 'user', 'content': [{'toolResult': {'toolUseId': 'tooluse_kuk3BxTXRgCW-WiQQriDAw', 'content': [{'json': {'result': [{'id': 424, 'quote': "I hated every minute of 
training, but I said, “Don't quit. Suffer now and live the rest of your life as a champion.”", 'author': 'Muhammad Ali'}, {'id': 1052, 'quote': 'How Hard, How Bitter It Is To Become A Man!', 'author': 'Albert Camus'}]}}]}}]}]
#### Final response
{
    "role": "assistant",
    "content": []
}

As you can see the response come back empty. From what i gathered here acceptable format for json is the follwing:

'json': {...}|[...]|123|123.4|'string'|True|None,

And i dont see anything wrong with my input.

What am I missing here?

asked 2 years ago558 views
1 Answer
0

Update: This could very well be a problem with our proxy or some other internal networking issue. Ran the code without any changes and got a response from Bedrock, ran it again and got a connection error to the proxy, ran it again and proxy was alive again but response came back empty as before.

Or its an intermittent issue with Bedrock. It comes back empty most of the time, but sometimes it works.

answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.