Skip to content

Previously working - Now having issues with Claude Sonnet using tools including knowledge base search and function calls

3

My Sonnet agent had been working well until roughly 24hrs ago. Then all of a sudden it lost the ability to search my knowledge base as well as having issues using other tools (Lambda functions). In an effort to find out what was going on I changed my agent to Titan Text G1 and I also tried Haiku. Both of them worked. I have also tried deleting data source, then knowledge base and starting fresh. No difference. To rule out any anomalies I might've unwittingly created, or general issues in N.Virginia region, I setup a barebones Sonnet agent with knowledge base in the Sydney region with the previously working data files. I am having the same issues with Sonnet in Sydney. Here is a trace from the agent when trying to call the knowledge base. It is supposed to lookup device IP addresses and management credentials. Basically it keeps saying "The function call format is incorrect". It also won't execute some other function calls that used to work. I have checked Permissions and policies. The prompts are defaults.

<tool_name>Okay, let me check the knowledge base for the management IP address of the device named \"cisco_r1\".<function_calls><invoke><tool_name>GET::x_amz_knowledgebase_VRVUWUVNFV::Search</tool_name><parameters><$PARAMETER_NAME>searchQuery</$PARAMETER_NAME>management IP address for device cisco_r1</parameters></tool_name><stdout>406: The function call format is incorrect. The format for function calls must be: <invoke> <tool_name>$TOOL_NAME</tool_name> <parameters> <$PARAMETER_NAME>$PARAMETER_VALUE</$PARAMETER_NAME>...</parameters></invoke>.</stdout></error></function_results>","role":"user"},{"content":"<thinking>Apologies, there seems to be an issue with the format of my function call. Let me try again with the correct format.</thinking><function_calls><function_calls><invoke><tool_name>GET::x_amz_knowledgebase_VRVUWUVNFV::Search</tool_name><parameters><$PARAMETER_NAME>searchQuery</$PARAMETER_NAME>management IP address for device cisco_r1</parameters></function_calls>","role":"assistant"},{"content":"<function_results><error><tool_name><function_calls><invoke><tool_name>GET::x_amz_knowledgebase_VRVUWUVNFV::Search</tool_name><parameters><$PARAMETER_NAME>searchQuery</$PARAMETER_NAME>management IP address for device cisco_r1</parameters></tool_name><stdout>406: The function call format is incorrect. The format for function calls must be: <invoke> <tool_name>$TOOL_NAME</tool_name> <parameters> <$PARAMETER_NAME>$PARAMETER_VALUE</$PARAMETER_NAME>...</parameters></invoke>.</stdout></error></function_results>","role":"user"},{"content":"<thinking>I apologize, I'm still having trouble with the proper format for invoking the knowledge base search function. Let me try one more time, carefully following the specified format.</thinking><function_calls><function_calls><invoke><tool_name>GET::x_amz_knowledgebase_VRVUWUVNFV::Search</tool_name><parameters><$PARAMETER_NAME>searchQuery</$PARAMETER_NAME>management IP address for device cisco_r1</parameters></function_calls>","role":"assistant"},{"content":"<function_results><error><tool_name><function_calls><invoke><tool_name>GET::x_amz_knowledgebase_VRVUWUVNFV::Search</tool_name><parameters><$PARAMETER_NAME>searchQuery</$PARAMETER_NAME>management IP address for device cisco_r1</parameters></tool_name><stdout>406: The function call format is incorrect. The format for function calls must be: <invoke> <tool_name>$TOOL_NAME</tool_name> <parameters> <$PARAMETER_NAME>$PARAMETER_VALUE</$PARAMETER_NAME>...</parameters></invoke>.</stdout></error></function_results>","role":"user"},{"content":"<thinking>I'm still unable to properly invoke the knowledge base search function after multiple attempts. To move forward, I will ask the user directly for the management IP address of the cisco_r1 device.

Thanks, Matt

3 Answers
0

Hi Matt,

as you have seen in my post before, I am experiencing the same problem.

In the response you posted, too, the queries the LLM formulated are faulty. They look just like the faulty queries my agents have started writing... As you mention, this hasn't been a problem before and just started two days back. Also, GPT-3 has long been able to handle similar tasks with ease. Really hope AWS fixes this ASAP!

To point out the problem even clearer if that isn't immediately obvious from your cited response:

<function_calls>
  <function_calls>
      <invoke>
       <tool_name>GET::x_amz_knowledgebase_VRVUWUVNFV::Search</tool_name>
       <parameters>
         <$PARAMETER_NAME>searchQuery</$PARAMETER_NAME>           // <-- $PARAMER_NAME-placeholders should prob be "searchQuery"
management IP address for device cisco_r1                          // <-- this string should prob be inside xml-tags, where "searchQuery" is
       </parameters>
    </function_calls>
                         // <-- the "invoke" tag isnt closed 
                         // <-- the (duplicate) first "function_calls" tag isnt closed               
answered 2 years ago
0

Ahh I hadn't bothered looking closely enough. Thanks Georg. The thing is, mine had been working quite well, although I often had to remind the agent to use the knowledge base which I thought was a bit weird, but otherwise everything was good. Just to let you know, when testing in AU region earlier today, I got three "internal server exception" errors in a row when asking Sonnet to search the knowledge base, after which it reverted to as above, again. Fred and Wilma will have something to say about this!

answered 2 years ago
0

Hi, I'm facing this exact issue with my agent. It adds multiple unnecessary <function_calls> opening tags causing the format error. It happens around 70% of the time and older versions of my agent seem to work ok. Anyone know of a fix?

answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.