How can I store the private vector data in Opensearch per user?

Question

Hi!

I'm building a RAG approach AI feature. I did a test with AWS Bedrock creating a knowledgebase that gets the data from a S3 bucket. Then the data is sent to opensearch and store as vectors.

I would like to know if there is the possibility of storing that data in Opensearch in a private way, so later each user can get the response of his own data from the chatbot UI Im building.
The user A will ask a question, and it should receive only the response based on his data, and then same should happen with the user B.

There is a way to achieve that? And is so how can I separate that data in Opensearch? I'm using Python with langchain.

Thank you guys!

Accepted Answer

Hi,

Rather than OpenSearch, did you explore the pgvector extension of PostgreSQL: it is available on AWS RDS. See https://aws.amazon.com/about-aws/whats-new/2023/05/amazon-rds-postgresql-pgvector-ml-model-integration/ or https://aws.amazon.com/about-aws/whats-new/2023/10/amazon-rds-postgresql-pgvector-hnsw-indexing/

This blog post will also detail the approach: https://aws.amazon.com/blogs/database/building-ai-powered-search-in-postgresql-using-amazon-sagemaker-and-pgvector/

The ability to add columns to your relational tables identifying different users will allow you to obtain the separation that you want.

Best,

Didier

How can I store the private vector data in Opensearch per user?

Relevant content