2개 답변
- 최신
- 최다 투표
- 가장 많은 댓글
0
You're correct that there's a limitation when using Amazon Bedrock's Web Crawler with Pinecone. Currently, the Web Crawler data source connector for Amazon Bedrock knowledge bases is only compatible with Amazon OpenSearch Serverless as the vector store. This is a specific limitation of the Web Crawler implementation.
If you want to use Pinecone as your vector store while still leveraging web content, you have a few options:
1. Use OpenSearch Serverless specifically for your web-crawled content: You could create a separate knowledge base using OpenSearch Serverless for web content, while using Pinecone for other data sources.
2. Create a custom ingestion pipeline: Instead of using Bedrock's Web Crawler, you could develop your own web scraping solution, process the content, and then load it into Pinecone directly. You would then connect this Pinecone vector store to your Bedrock knowledge base.
3. Use a different supported data source type with Pinecone: While Web Crawler specifically requires OpenSearch Serverless, other data source types (like S3) should work with Pinecone as your vector store.
Keep in mind that vector store options for Amazon Bedrock Knowledge Bases include Amazon OpenSearch Serverless, Amazon Aurora PostgreSQL, Amazon Neptune Analytics, and third-party options like Pinecone, Redis Enterprise Cloud, and MongoDB Atlas - but the Web Crawler specifically has the OpenSearch Serverless requirement.
Sources
Bedrock Knowledge base with Normal PostgreSQL Vector DB (without Aurora) | AWS re:Post
Vector store options for Bedrock Knowledgebase | AWS re:Post
Crawl web pages for your knowledge base - Amazon Bedrock
0
Hello.
I tried it with my AWS account, but I was unable to set up the Web Crawler with Pinecone.
As of May 2025, it appears that if you want to use the Web Crawler, you will need to use OpenSearch Serverless.
Therefore, there is no option to use it with Pinecone.
https://docs.aws.amazon.com/bedrock/latest/userguide/webcrawl-data-source-connector.html
