Skip to content

Access Denied Exception when using CreateDataSourceCommand

0

Howdy,

I've got a pretty frustrating issue I can't solve. I'm trying to use the SDK to:

  1. Create an S3 bucket for a user (success!)
  2. Create an S3 vector database for a user (success!)
  3. Create a knowledge base for a user using the S3 vector database (success!)
  4. Use the S3 bucket in step one as the datasource for the knowledge base (failure)

When I get to step 4, the error is some flavor of: 'User: arn:aws:iam::<account_id>:user/test-bedrock is not authorized to perform: bedrock:CreateDataSource', even though that user has performed steps 1 through 3 with the correct permissions.

What I've tried:

  1. Confirm that I can use the programatically generated bucket, s3 vector database, and knowledgebase by using the Web Interface to manually create and add a data source to the knowledgebase.
  2. Ensuring that permissions are correct. The user has full S3 and Bedrock permissions. I have confirmed in the IAM policy simulation that the user has bedrock:CreateDataSource permission. Enter image description here
  3. When I create the knowledgebase, it is using a role called BedrockAgentRole. I've confirmed that this role also has full access, and that the user test-bedrock can correctly assume the role of BedrockAgentRole, which is required for creating the Knowledge Base in step 2. Both seem to have full S3 and Bedrock access, at least to perform steps 1 through 3 above.
  4. Switched credentials to the service role that created the knowledge base to see if that works, the error in that case is `'User: arn:aws:sts::<account_id>:assumed-role/BedrockAgentRole/AWSPlaygroundSession is not authorized to perform: bedrock:CreateDataSource'. Again, both the user and role appear to have full S3 and Bedrock permissions.

My code to create the Data Source programmatically is:

    try{
      response = await bedrockClient.send(new CreateDataSourceCommand({
        knowledgeBaseId: `kb-${user.id}`,
        name: `ds-${user.id}`,
        dataSourceConfiguration: {
          type: "S3",
          s3Configuration: {
            bucketArn: `arn:aws:s3:::${process.env.PRIVATE_AWS_S3_BUCKET}`,
            inclusionPrefixes: [
              `kb/${user.id}/`
            ]
          },    
        }
      }));
      console.log('Data source created:', response.$metadata);
    } catch(e) {
      console.error("Error creating data source:", e);
      throw e;
    }
2 Answers
1
Accepted Answer

Hello.

As mentioned in the following stackoverflow answer, an error may also occur if the SDK parameters are missing.
Therefore, I recommend that you create a knowledge base from the management console, check CloudTrail, and look for differences with requests made via the SDK.
https://stackoverflow.com/a/79056358

Also, check whether you can create a data source by setting Administrator as an IAM policy.
By the way, is your AWS account managed by AWS Organizations?
If so, it's possible that the request is being denied by an SCP.

EXPERT

answered a year ago

EXPERT

reviewed 2 months ago

  • Hi Riku,

    That was an awesome idea about looking at CloudTrail and seeing what successful parameters look like, a trick I will for sure use in the future. Unfortunately, even after putting parameters the same, I am still getting an AccessDeniedException. The user in the successful case is the same user that is making the SDK call, except using Access Keys.

    This AWS account is not managed by AWS Organizations.

    The event source for the request comes from bedrock.amazonaws.com, does my bucket need additional permissions? I don't have ACL enabled

  • Ok, I feel really dumb. I was using the knowledge base name and not the ID. Thank you for the suggestion to use the CloudTrail as a way to validate successful web UI calls to services that can be integrated into the SDK calls. I will mark that as the correct answer.

0

The AccessDeniedException you're encountering when trying to create a data source for your knowledge base is likely due to missing specific permissions required for the CreateDataSource operation in Amazon Bedrock.

Based on your description, here are some potential causes and solutions:

  1. Missing specific Bedrock permissions: While you mentioned having "full Bedrock permissions," make sure the IAM policy explicitly includes bedrock:CreateDataSource. Sometimes "full access" managed policies might not include all the newest API actions.

  2. Service role permissions: When creating a knowledge base, Amazon Bedrock creates service-linked roles with specific naming conventions. The error suggests that either your user or the BedrockAgentRole doesn't have the exact permissions needed. The service-created policies often have paths like "service-role/" in their ARNs.

  3. Resource-specific permissions: Your policy might need to specify the exact resource ARN for the knowledge base you're trying to modify. Check if your policy includes the specific knowledge base resource.

  4. Cross-service permissions: Creating a data source involves both Bedrock and S3 services. Ensure your policy includes permissions for both services and any cross-service actions.

  5. Explicit denies: Check if there are any explicit deny statements in other policies that might be overriding your allow statements. This could be in resource control policies, service control policies, or permission boundaries.

Try updating your IAM policy to explicitly allow the CreateDataSource action on the specific knowledge base resource. If you're using an organization with SCPs (Service Control Policies), also check that they're not restricting this action.
Sources
CreateDataSource - Amazon Bedrock
Unable to create data source from S3 bucket for Knowledge Base using boto3 | AWS re:Post
Permission issues while creating knowledge base in bedrock | AWS re:Post
Troubleshoot access denied (403 Forbidden) errors in Amazon S3 - Amazon Simple Storage Service

answered a year ago

EXPERT

reviewed a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.