Skip to content

AWS Bedrock knowledge base for Structured Data with Glue Tables

0

Does AWS support using structured data retrieval in Amazon Bedrock Knowledge Bases with Glue Database? We use virtual table AWS Glue to catalog our tables connected to the S3. When trying to create knowledge base, it only list Redshift for structured data source. Are you planning to support Glue tables in the future? We use S3 >> Glue >> Athena for analytical processing. We wish if we can use knowledge base to query the Glue tables like in Redshift.

asked 2 years ago1.9K views

2 Answers
1

Yes, AWS does support using structured data retrieval in Amazon Bedrock Knowledge Bases with AWS Glue Data Catalog, which includes Glue databases and tables. While the primary interface may prominently display Amazon Redshift as an option, AWS Glue Data Catalog is indeed a supported data source for structured data in Amazon Bedrock Knowledge Bases.

When creating a knowledge base for structured data, you can connect to AWS Glue Data Catalog (which is part of AWS Lake Formation). This means you can use your virtual tables in AWS Glue that catalog your S3 data for querying through Amazon Bedrock Knowledge Bases.

To use Glue tables in your knowledge base:

  1. When setting up the knowledge base, you should be able to select AWS Glue Data Catalog as your data source.
  2. You'll need to ensure that the Amazon Bedrock service role has the necessary permissions to access your Glue Data Catalog. This includes permissions like glue:GetDatabases, glue:GetTables, and others.
  3. When specifying table names, you'll need to use the format awsdatacatalog.gluedatabase.table to reference your Glue tables.

It's important to note that while you're using Glue tables, the query engine behind the scenes is still Amazon Redshift. Amazon Bedrock Knowledge Bases uses the Amazon Redshift query engine to connect to and query data from supported structured data stores, including AWS Glue Data Catalog.

This setup should allow you to use Amazon Bedrock Knowledge Bases to query your Glue tables in a similar way to how you would query Redshift tables. You can use natural language queries, which will be converted into SQL queries appropriate for your data structure.

If you're experiencing issues seeing or connecting to your Glue tables, it may be worth double-checking your permissions and configurations, or reaching out to AWS support for assistance.
Sources
Turning data into a knowledge base - Amazon Bedrock
Build a knowledge base by connecting to a structured data store - Amazon Bedrock
Create a knowledge base by connecting to a structured data store - Amazon Bedrock
Allow your Amazon Bedrock Knowledge Bases service role to access your data store - Amazon Bedrock

answered 2 years ago

EXPERT

reviewed 2 years ago

0

However, on next step when choosing Glue Catalog generates error "Redshift Workgroup is required". I do not use Redshift.

answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.