Skip to content

Bedrock has no general knowledge and doesn't understand questions asked in different ways

0

Hi all,

I am building a generative AI chatbot using AWS Bedrock, AWS OpenSearch Serverless, AWS Lambda, AWS CloudFormation, and AWS S3. Here is the GitHub repository for the code https://github.com/aws-samples/amazon-bedrock-samples/tree/main/rag-solutions/contextual-chatbot-using-knowledgebase

I have tested the chatbot using my own data file. However, it only answers questions that are phrased in a certain way (ie questions that use words that are also used in the data file). For example, my document talks about the feedback my company has received from potential customers. When I ask the chatbot "what feedback have potential customers given my company?", the chatbot responds correctly. But when I ask the chatbot "what have potential customers said about my company?", the chatbot responds with "Sorry, I am unable to assist you with this request".

How do I get the chatbot to understand that questions rephrased differently require the same information from my data file?

Also, I asked the chatbot general knowledge questions (so information not included in my data file, but general questions like "why is the sky blue?") but the chatbot also returned "Sorry, I am unable to assist you with this request". I am confused as I am using the Claude 3.5 Sonnet LLM, so thought the chatbot would be able to answer generic queries that aren't related to the data I am feeding the bot via my S3 bucket. How do I activate the general knowledge capability within the bot, given that Claude 3.5 Sonnet is trained on public data from all over the web and should be able to answer easy questions like that?

Thank you

asked a year ago390 views
1 Answer
3
Accepted Answer

Hello!!

The issues you're facing with AWS Bedrock might be related to how your data is ingested and managed within the knowledge base. Ensure that your data source, such as S3, is correctly synced and that the embeddings are properly configured. Bedrock relies on correct chunking and metadata to handle queries effectively. For general knowledge capabilities, ensure that the model configuration allows fallback to broader datasets beyond your custom knowledge base.

For more details, refer to the syncing your data source and S3 data source connection documentation. I'm here to help.

EXPERT
answered a year ago
EXPERT
reviewed a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.