I'm struggling to understand what's going on here, so just asking for insights or help :)
I'm trying to implement RAG using a Faiss vector store on nodejs running against the titan express FM.
However the results I'm getting are not dependable in the simple case, and non-existent when trying to submit context with my query. If I directly invoke the model I usually get a sensible result but even that often gives "Sorry - this model is unable to respond to this request." with exactly the same query:
$ node index-4.js
Query Amazon Bedrock:
question: 'what does article 25 of the gdpr say?'
Direct invocation: Article 25 of the GDPR states that the data subject has the right to request the erasure of his or her personal data from the controller without undue delay. The controller shall have the right to determine the time limits for erasure.
Individual steps: Sorry - this model is unable to respond to this request.
Use VectorDBAQChain: Sorry - this model is unable to respond to this request.
$ node index-4.js
Query Amazon Bedrock:
question: 'what does article 25 of the gdpr say?'
Direct invocation: Sorry - this model is unable to respond to this request.
Individual steps: Sorry - this model is unable to respond to this request.
Use VectorDBAQChain: Sorry - this model is unable to respond to this request.
Any suggestions for how I can debug this code? I don't know what's flakey here, langchain or the FM:
async function queryBedrock(vs_filename, docs, question)
{
const model = new Bedrock({
model:"amazon.titan-text-express-v1",
region: process.env.BEDROCK_REGION,
credentials: {
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
},
temperature: 0.7,
maxTokens: 2500
});
/////////////////////////////
// invoke model directly
/////////////////////////////
try {
let result = await model.invoke(question);
console.log(`Direct invocation: ${result}`);
}
catch (e){
console.log(e);
}
/////////////////////////////
// use individual steps
/////////////////////////////
const embeddings = new BedrockEmbeddings({
model: "amazon.titan-embed-text-v1",
region: process.env.BEDROCK_REGION,
credentials: {
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
},
});
let vectorStore = await FaissStore.load(vs_filename, embeddings);
const retriever = vectorStore.asRetriever();
const prompt = PromptTemplate.fromTemplate(`Answer the question based only on the following context: {context} Question: {question}`);
const indiv_chain = RunnableSequence.from([
{
context: retriever.pipe(formatDocumentsAsString),
question: new RunnablePassthrough(),
},
prompt,
model,
new StringOutputParser(),
]);
const result = await indiv_chain.invoke(question);
console.log(`Individual steps: ${result}`);
/////////////////////////////
// use VectorDBQAChain
/////////////////////////////
const dbqa_chain = VectorDBQAChain.fromLLM(
model,
vectorStore,
{ k: 10, returnSourceDocuments: true }
);
const res = await dbqa_chain.call({
input_documents: docs,
query: question
});
console.log(`Use VectorDBAQChain: ${result}`);
}