Amazon Q Business Get Metadata info from S3 bucket specific folder

To get metadata information from specific folders in your S3 bucket for use with Amazon Q Business, you can configure your data source to focus on those particular locations. Here's how you can approach this:

1. When creating your Amazon S3 data source for Amazon Q Business, you can specify inclusion prefixes to target specific folders. In your case, you would want to include the paths to your 'received' folders.

2. You can set up your data source configuration using either the AWS Management Console or the CreateDataSource API with a JSON schema. If using the API, you would include an 'inclusionPrefixes' field in your configuration.

3. For example, your configuration might include:

   "inclusionPrefixes": ["/feeds/1100/received/", "/feeds/1200/received/"]

   This tells Amazon Q Business to look only in these specific folders for documents to index.

4. To ensure that new files are reflected when added, you should set the 'syncMode' to "INCREMENTAL_CRAWL". This allows Amazon Q Business to periodically check for new or updated files and index them accordingly.

5. For the file types you mentioned (.zip, .txt, .json), you can use inclusion patterns to specify which file types to index. For example:

   "inclusionPatterns": [".txt", ".json"]

   Note that .zip files would need to be extracted before indexing.

6. To add metadata to your files, you can create metadata files in the same S3 bucket. These should be JSON files with a specific structure, including fields like DocumentId, Attributes, Title, and ContentType. The metadata files should be in a parallel directory structure to your documents or in the same location if you don't specify a separate prefix.

7. If you want to specify a separate location for metadata files, you can use the 'metadataFilesPrefix' field in your configuration. For example:

   "metadataFilesPrefix": "/metadata/"

By setting up your data source this way, Amazon Q Business will index the files in your specified 'received' folders, including any new files added over time, and incorporate any metadata you provide. This will allow users to ask questions about the content and metadata of these files through the Amazon Q Business application.
Sources
Adding document metadata in Amazon S3 - Amazon Q Business
Connecting Amazon Q Business to Amazon S3 using APIs - Amazon Q Business
Answer questions from tables embedded in documents with Amazon Q Business | AWS Machine Learning Blog

Hello,

Thank you for reaching out to us.

I understand that you would like to get the metadata information from specific s3 folder. I would like to mention that you can use Lambda functions to prepare your document attributes for advanced data manipulation.

You can choose to apply a basic operation first and then use a Lambda function to manipulate your data, and the reverse. With the code of the lambda function you can also get the metadata as well. https://docs.aws.amazon.com/amazonq/latest/qbusiness-ug/cde-lambda-operations.html#cde-lambda-operations-data-contracts

I hope the above information is helpful. Please feel free to get back if you have any further query in this regard. I will be happy to address them.

Thank you and have a nice day!