Skip to content

Vector store options for Bedrock Knowledgebase

0

I used Bedrock workshop example to create a knowledgebase which used OpenSearch serverless as the vector store. At the end of month, this came with a bill of $260 because apparently it allocates minimum 4 OCUs.

To reduce the cost, I am looking for vector store options other than OpenSearch serverless, which Bedrock Knowledgebase is compatible with. My first preference is to use pgvector extension in RDS PostgreSQL which is not supported (though Aurora PostgreSQL is, but I do not want to use that). I thought another option would be to use a managed instance of OpenSearch (instead of Serverless).

However, in the form to create new KnowledgeBase, under the "Vector Database" section, I can only see two possibilities: 1- Quick Create a new Vector Store -> Amazon OpenSearch Serverless (which is what I am trying to avoid) 2- Choose a vector store you have created -> Vector engine for Amazon OpenSearch Serverless

When I proceed with the option 2, it asks for a collection ARN which is only supported by the serverless version of OpenSearch.

Enter image description here

In fact, on the Comparing OpenSearch Service and OpenSearch Serverless page, it is listed that "collections" are only supported in OpenSearch serverless version, not OpenSearch service. However, it does not say that vectors cannot be used in OpenSearch Service.

Is my only option to re-create the OpenSearch Serverless collection with modified settings? Can I really not use RDS Postgresql using pgvector or a small opensearch on-demand instance as my vector store for the purpose of using Bedrock knowledgebase?

2 Answers
0

Unfortunately, your options for vector stores with Amazon Bedrock Knowledge Bases are indeed limited, and using a standard RDS PostgreSQL with pgvector or a managed OpenSearch instance is not currently supported.

For Amazon Bedrock Knowledge Bases, the supported vector store options are:

  1. Amazon OpenSearch Serverless
  2. Amazon Aurora PostgreSQL-Compatible Edition
  3. Amazon Neptune Analytics
  4. Some third-party vector stores like Pinecone, Redis Enterprise Cloud, and MongoDB Atlas

The interface you're seeing in the Knowledge Base creation form reflects these limitations. The "Quick Create" option using Amazon OpenSearch Serverless is designed for ease of setup, while the "Choose a vector store you have created" option is specifically for OpenSearch Serverless collections, as indicated by the ARN requirement.

While Aurora PostgreSQL is supported and could potentially be a more cost-effective option than OpenSearch Serverless, you mentioned you don't want to use it. Unfortunately, standard RDS PostgreSQL with pgvector is not a supported option for Bedrock Knowledge Bases, even though it can be used as a vector store in other contexts.

Regarding your concern about the cost of OpenSearch Serverless, you're correct that it allocates a minimum of 4 OCUs. To reduce costs, you have a few options:

  1. Reconfigure your OpenSearch Serverless collection to optimize resource usage.
  2. Consider using Aurora PostgreSQL Serverless v2, which can scale down to zero when not in use, potentially helping with costs.
  3. Explore the third-party vector store options if they align with your needs and budget.

If none of these options work for you, you may need to reconsider your approach or explore alternative solutions outside of Bedrock Knowledge Bases for your vector search needs. However, this would mean losing the integrated capabilities that Bedrock Knowledge Bases provide for Retrieval Augmented Generation (RAG) workflows.
Sources
Bedrock Knowledge base with Normal PostgreSQL Vector DB (without Aurora) | AWS re:Post
Knowledge bases for Amazon Bedrock - AWS Prescriptive Guidance
Amazon Aurora now available as a quick create vector store in Amazon Bedrock Knowledge Bases - AWS

answered 10 months ago
EXPERT
reviewed 10 months ago
0
answered 5 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.