Problems using langchain and Bedrock with titan FM

0

I'm struggling to understand what's going on here, so just asking for insights or help :)

I'm trying to implement RAG using a Faiss vector store on nodejs running against the titan express FM.

However the results I'm getting are not dependable in the simple case, and non-existent when trying to submit context with my query. If I directly invoke the model I usually get a sensible result but even that often gives "Sorry - this model is unable to respond to this request." with exactly the same query:

$ node index-4.js
Query Amazon Bedrock:
question: 'what does article 25 of the gdpr say?'
Direct invocation: Article 25 of the GDPR states that the data subject has the right to request the erasure of his or her personal data from the controller without undue delay. The controller shall have the right to determine the time limits for erasure.
Individual steps: Sorry - this model is unable to respond to this request.
Use VectorDBAQChain: Sorry - this model is unable to respond to this request.

$ node index-4.js
Query Amazon Bedrock:
question: 'what does article 25 of the gdpr say?'
Direct invocation: Sorry - this model is unable to respond to this request.
Individual steps: Sorry - this model is unable to respond to this request.
Use VectorDBAQChain: Sorry - this model is unable to respond to this request.

Any suggestions for how I can debug this code? I don't know what's flakey here, langchain or the FM:

async function queryBedrock(vs_filename, docs, question)
{   
    const model = new Bedrock({
        model:"amazon.titan-text-express-v1",
        region: process.env.BEDROCK_REGION,
        credentials: {
            accessKeyId: process.env.AWS_ACCESS_KEY_ID,
            secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
        },
        temperature: 0.7,
        maxTokens: 2500
    });

    /////////////////////////////
    // invoke model directly
    /////////////////////////////

    try {
        let result = await model.invoke(question);
        console.log(`Direct invocation: ${result}`);
    }
    catch (e){
        console.log(e);
    }

    /////////////////////////////
    // use individual steps
    /////////////////////////////

    const embeddings = new BedrockEmbeddings({
        model: "amazon.titan-embed-text-v1",
        region: process.env.BEDROCK_REGION,
        credentials: {
            accessKeyId: process.env.AWS_ACCESS_KEY_ID,
            secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
        },
    });
    
    let vectorStore = await FaissStore.load(vs_filename, embeddings);
    const retriever = vectorStore.asRetriever();
    const prompt = PromptTemplate.fromTemplate(`Answer the question based only on the following context: {context} Question: {question}`);
    const indiv_chain = RunnableSequence.from([
                    {   
                        context: retriever.pipe(formatDocumentsAsString),
                        question: new RunnablePassthrough(),
                    },
                    prompt,
                    model,
                    new StringOutputParser(),
                ]);
    
    const result = await indiv_chain.invoke(question);
    console.log(`Individual steps: ${result}`);

    /////////////////////////////
    // use VectorDBQAChain
    /////////////////////////////

    const dbqa_chain = VectorDBQAChain.fromLLM(
                        model,
                        vectorStore,
                        { k: 10, returnSourceDocuments: true }
                    );

    const res = await dbqa_chain.call({
        input_documents: docs,
        query: question
    });
    console.log(`Use VectorDBAQChain: ${result}`);
}
dmb0058
질문됨 6달 전813회 조회
2개 답변
0

Enable logging on Bedrock in order to see which prompts are being sent to the model. This will allow you to determine if the problem lies in the retrieval, eg the context passed in the prompt is not existent or wrong, or in the model. Once you have the prompts, try to execute them directly to see if there is flakiness or if they are systematically returning no answers. You can also try to manually modify such prompts in order to try to get the answers and then make the necessary adjustments to your solution.

AWS
전문가
답변함 6달 전
0

Great idea! I'm only starting with Bedrock and disn't realise it had logging so this is really useful thanks.

I've established that it is highly variable (same prompt may or may not give a result) and extremely prompt-sensitive (slight change of context such as "... in no fewer than 450 words ..." vs "... in no fewer than 500 words ..." makes a significant difference) compared to OpenAI, so I've managed to get some good output but your tips will help me to figure out what's working and what isn't.

dmb0058
답변함 6달 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠