I'm using a Sonnet 3 agent to make various API calls to third party systems. I am finding it's ability to invoke the tools for the various API's very inconsistent - and frustrating, e.g. works one day and not the next. On tracing I have found that it randomly varies how it inserts parameters into the XML tags and constructs XML in general, e.g. it sometimes inserts the parameters into the path in the <tool_name> tags (incorrect) while sometimes it doesn't. It also seems to neglect it's accompanying schema and will use <tool_name>magical-mystery-tool/path</tool_name> instead of <tool_name>get::magical-mystery-tool::/path</tool_name>
Often I have success by telling the agent what it's doing wrong and giving it the XML example it needs to follow even though this is already correctly defined in the Orchestration prompts so I shouldn't have to do this and can't expect users to. Maybe I need to do all this in code rather than using the Console?
Is there any way to improve this/make it more consistent?
Thanks :-)
The issue looks to be with the tool calling itself, not the code execution. So I think this means it's more related to agents interpretation of the OpenAPI schema. The thing is it can work perfectly one time and not the next. I have managed to workaround it to some degree by providing instructions within the schema itself on how to format the XML.