- Newest
- Most votes
- Most comments
Hi,
Knowledge Bases for Bedrock use the standard Bedrock LLMs under the hood (you indeed choose which one when configuring your KB).
So, you should try to use so-called "Model invocation logging" to intercept requests and responses: https://docs.aws.amazon.com/bedrock/latest/userguide/model-invocation-logging.html will tell you how to implement it.
It will work if KB re-use the standard Bedrock API from your account and doesn't go direct to some internal LLM API: I didn't try with KB but I would guess that it uses the standard APIs.
Then, in CloudWatch logs or S3, you will see the full thing as per doc:
With invocation logging, you can collect the full request data, response data, and
metadata associated with all calls performed in your account. Logging can be configured
to provide the destination resources where the log data will be published. Supported destinations include
Amazon CloudWatch Logs and Amazon Simple Storage Service (Amazon S3). Only destinations from
the same account and region are supported.
Best
Didier
Hello.
How about registering the message sent to the knowledge base and the response obtained by RetrieveAndGenerate to DynamoDB or RDS?
In other words, if you add processing to register in DynamoDB etc. in the code that executes RetrieveAndGenerate, you can record conversations etc.
https://docs.aws.amazon.com/bedrock/latest/userguide/kb-test-how.html
Relevant content
- asked 6 months ago
- asked 9 months ago
- asked 3 months ago
AWS OFFICIALUpdated 4 months ago- AWS OFFICIALUpdated 4 months ago
- AWS OFFICIALUpdated 4 months ago
Thank you! I came across this article after I posted and created the Cloudwatch logs: https://aws.amazon.com/blogs/mt/monitoring-generative-ai-applications-using-amazon-bedrock-and-amazon-cloudwatch-integration/
Now I'm seeing that I can't use the same query that is in the example because the UserInput object isn't available. I can see the user input is recorded and provided but i think it is provided as token vectors and not the actual text. Have you been able to get this working on your end?
Thanks for accepting my answer! I'll try on my side and let you know.
Hi JRath,
I now made tests on my side to confirm (I used logging to S3 in addition to CW). So, for each request, I receive a json file containing 2 independent requests, each in a separate json object. As you pointed out, the first one is the embedding answer of the question text from the embedding engine selected for the KB (Cohere v3 English in my case) and the second json object is a call to the selected LLM with the initial question + a system prompt with the RAG response of the KB with the chunks of knowledge in you data source corresponding to the embeddings of the question.
So, Bedrock KB is fully observable in this manner. Thanks for having pushed me to make the full investigation! It was interesting
Thank you for your help! Good call to go directly to the log file itself. I was trying to view via Cloudwatch where I couldn't see the distinction between call to the embedding vs call to the LLM. Much appreciated!