Bedrock KnowledgeBase sync error, can't access S3 bucket
I'm following the instructions here to create a Knowledge Base.
I configure the Knowledge Base to have on of my S3 buckets (under the same account) as my data source. However, when I try to Sync the data source, it errors out right away with:
Encountered error: Access Denied (Service: S3, Status Code: 403, Request ID: JDKB992YNZEJ475K, Extended Request ID: VhXJ7Nrt+ZYjxx5IlAnwsg+Wmu2iLGeJ/7zomRMhr0OW83Sac+BtiwUMpe+9XYj0+zkwl8LMau0=). Call to Amazon S3 Source did not succeed.
The Role/policies that the console setup seem right.
Any thoughts about what could be wrong here?
- Newest
- Most votes
- Most comments
The policy generated by Bedrock when creating the Knowledge Base data source is incorrect at the time being - it lacks reference to the bucket itself in Resources so as to authorize the ListBucket action itself. Manually adding the bucket to the Resources (in your case "arn:aws:s3:::equilo-data-ingestion") will solve the issue. NB: I would recommend to hide your account number when posting in an open forum for security.
Please check the region in which your bucket has been created and the region where bedrock is being used. I had the same issue, just checked the regions between both and it is resolved. The autogenerated policies have proper access you do not need to edit anything else.
Hey, couple of reasons that i could assume causing the above failure.
- Proper IAM polices are not setup.
s3:GetObject,s3:ListObjectare the bare minimum policies to copy over the S3 bucket object. But if your bucket is enabled with versioning, you needs3:GetObjectVersionpermission. And to copy the objects with tags, you needs3:GetObjectTagging(source bucket),s3:PutObjectTagging(needed for destination bucket). If you have attached the permission polices to the destination bucket, make sure that bucket has these policiess3:GetObject,s3:PutObject, ands3:ListBucket. - S3-SSE. If you have enabled the Server-side encryption with AWS Managed KMS Key or Customer Managed Key, you should have
kms:Decrypt,kms:GenerateDataKeypermissions on specific KMS key in resource section of IAM policy.
Thanks for the reply. Good to know about the additional permissions needed in these various scenarios, which I am about to get into. For now, the issue was about the actual bucket not being listed as a resource in the policy, even though I only wanted to ingest files in a sub folder/object.
I am getting similar sync error, but it says Knowledge base role is not able to call specified embedding model . But I see the policy is generated. What could be the issue?
Relevant content
- asked a year ago
AWS OFFICIALUpdated 9 months ago- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated 2 years ago
First, thanks for the security callout. That was an oversight. I have updated the screenshot in the question, fwiw.
You were totally right. Adding the bucket itself as a resource to the policy enabled a sync. Hope the default policy generation can account for this edge case where a user only wants to ingest a buckets sub folder/object.
Seems fixed now.