What is the character limit on on the metadata attributes in a metadata file of AWS Bedrock Knowledge Base?

Question

We have created a Knowledge Base with a mongodb vector DB and a storage layer of S3. 
We have metadata files for each file that we have (both are in JSON format, the data file and the metadata file).
However, sometimes we the below error: -

Encountered error: Ignored 1 files due to invalid metadata attributes. Check that the attribute keys and values don\'t exceed the character quota, and that the attribute values are acceptable data types (strings, numbers, or Booleans). Then retry your request [Files: s3://#####-bucket/#######/folder/file.json]. Call to Customer Source did not succeed."

We already know that we have to keep the metadata file's size below 10KB and that the acceptable data types are boolean, string, double, and integer . but we are still getting this error. There is no official documentation on the character limit.
We searched here
1. https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base-ds.html
2. https://docs.aws.amazon.com/bedrock/latest/userguide/quotas.html

Can you please guide us on the exact character limit for each field

Thanks,

Didier Durand · Answer

Hi,

Your question doesn't seem to have a direct answer but at least it has an indirect one.

If you go to https://docs.aws.amazon.com/bedrock/latest/userguide/quotas.html

In section "Knowledge base quotas", you'll see:

```
Ingestion job file size (metadata file) = 10 KB - The maximum size (in KB) of a metadata file 
in an ingestion job.
```

So, this gives you a max max size for any attribute.

What I usually do in this case is use trial and error:  I'd try with 1 file with 1 att slightly over 10 KB to see the error message that I obtain. Those error messages will most often tell the max size allowed for a field exceeding maximum size.

Best,

Didier

What is the character limit on on the metadata attributes in a metadata file of AWS Bedrock Knowledge Base?

Add your answer

Relevant content