How can I store the private vector data in Opensearch per user?

Hi!

I'm building a RAG approach AI feature. I did a test with AWS Bedrock creating a knowledgebase that gets the data from a S3 bucket. Then the data is sent to opensearch and store as vectors.

I would like to know if there is the possibility of storing that data in Opensearch in a private way, so later each user can get the response of his own data from the chatbot UI Im building. The user A will ask a question, and it should receive only the response based on his data, and then same should happen with the user B.

There is a way to achieve that? And is so how can I separate that data in Opensearch? I'm using Python with langchain.

Thank you guys!

トピック

サーバーレス分析機械学習と AI AWSでのジェネレーティブAI

タグ

Amazon OpenSearch Service Amazon Bedrock AWSでのジェネレーティブAI ラングチェイン

言語

English

cao95

質問済み 4ヶ月前845ビュー

1回答

新しい順
投票が多い順
コメントが多い順

これらの回答は役に立ちましたか？コミュニティがあなたの知識を活用できるように、正解に賛成票を投じてください。

承認された回答

Hi,

Rather than OpenSearch, did you explore the pgvector extension of PostgreSQL: it is available on AWS RDS. See https://aws.amazon.com/about-aws/whats-new/2023/05/amazon-rds-postgresql-pgvector-ml-model-integration/ or https://aws.amazon.com/about-aws/whats-new/2023/10/amazon-rds-postgresql-pgvector-hnsw-indexing/

This blog post will also detail the approach: https://aws.amazon.com/blogs/database/building-ai-powered-search-in-postgresql-using-amazon-sagemaker-and-pgvector/

The ability to add columns to your relational tables identifying different users will allow you to obtain the separation that you want.

Best,

Didier

エキスパート

Didier_Durand

回答済み 4ヶ月前

エキスパート

Sedat Salman

レビュー済み 4ヶ月前

cao95
4ヶ月前
Hey! An update here. I used this approach and it worked as I needed, thank you!
cao95
4ヶ月前
Hi again! I was doing some tests and I loaded a very long excel file into my pgvector postgresql database. Then I did a query to get the embeddings by the user id, and got the response, but when calling bedrock I got an exception. ValueError: Error raised by bedrock service: An error occurred (ValidationException) when calling the InvokeModel operation: Input is too long for requested model.

How can I manage large volume of data with this approach using RAG?

How can I store the private vector data in Opensearch per user?

関連するコンテンツ