How do I troubleshoot "403" or "401" errors that I receive when I connect to my OpenSearch Serverless cluster or dashboards?

Resolution

Note: If you receive errors when you run AWS Command Line Interface (AWS CLI) commands, then see Troubleshooting errors for the AWS CLI. Also, make sure that you're using the most recent AWS CLI version.

"403 Forbidden" errors

A "403 Forbidden" error message usually occurs when the AWS Identity and Access Management (IAM) identity doesn't have the correct permissions. "403 Forbidden" errors might also occur if the IAM identity has an incorrectly signed request.

When you turn on verbose logging in your client or choose the Network tab on the IAM console, you might get the following error:

"x-aoss-response-hint': 'X01:gw-helper-deny"

For OpenSearch collection dashboards and endpoint access, include the aoss:DashboardsAccessAll and aoss:APIAccessAll actions in the IAM identity's permissions policy. To view an example policy, see Using OpenSearch API operations.

For OpenSearch Serverless collection endpoint access, an incorrectly signed request can also cause a "403 Forbidden" error. Confirm that your IAM permissions are correct, and then review how your client signs requests and sends them to collection endpoint.

To write a basic version of your code, see Ingesting data into Amazon OpenSearch Serverless collections.

Use curl to test your endpoint from an Amazon Elastic Compute Cloud (Amazon EC2) instance.

Example curl command that ingests a record into an index:

curl -XPOST \
    --user "$AWS_ACCESS_KEY_ID":"$AWS_SECRET_ACCESS_KEY" \
    --aws-sigv4 "aws:amz:us-east-1:aoss" \
    --header "x-amz-content-sha256: $REQUEST_PAYLOAD_SHA_HASH" \
    --header "x-amz-security-token: $AWS_SESSION_TOKEN" \
    "https://my-collection-endpoint.us-east-1.aoss.amazonaws.com/movies-index/_doc" \
    -H "Content-Type: application/json" -d "$PAYLOAD"

Note: Replace the AWS Region, access key ID, secret key, and session token with your values. To get these values, run the get-session-token AWS CLI command.

To verify the current IAM identity and confirm that it matches a principal in your AOSS data access policy, complete the following steps:

1. Run the get-caller-identity command:

   aws sts get-caller-identity

   Example output:

   {
       "UserId": "AIDEXAMPLE123",
       "Account": "123456789012",
       "Arn": "arn:aws:iam::123456789012:user/my-aoss-user"
   }

2. Generate temporary credentials:

   aws sts get-session-token

   Example output:

   {
       "Credentials": {
           "AccessKeyId": "XXXXXXXXXXXX",
           "SecretAccessKey": "YYYYYYYYYYYYY",
           "SessionToken": "ZZZZZZZZZZZZZZZZ...",
           "Expiration": "1900-01-01T00:00:00Z"
       }
   }

3. Set the environment variables:

   export AWS_ACCESS_KEY_ID="XXXXXXXXXXXX"
   export AWS_SECRET_ACCESS_KEY="YYYYYYYYYYYYY"
   export AWS_SESSION_TOKEN="ZZZZZZZZZZZZZZZZ..."

4. (Optional) If you send a payload, then run the following command to generate a Secure Hash Algorithm 256-bit (SHA-256) of the payload as:

   export PAYLOAD='{"title": "Shawshank Redemption"}'
   
   # Generate SHA-256 hash (awk extracts only the hash value part)
   export REQUEST_PAYLOAD_SHA_HASH=$(echo -n "$PAYLOAD" | shasum -a 256 | awk '{print $1}')

You can also use the awscurl command on the GitHub website. The awscurl command uses your default credentials, such as your EC2 instance profile, so that you don't need to hash the payload as a header.

Example awscurl requests:

awscurl --service aoss --region us-east-1 -XPOST \
    "https://my-collection-endpoint.us-east-1.aoss.amazonaws.com/movies-index/_doc" \
    -H "Content-Type: application/json" -d '{"title": "Shawshank Redemption"}'

awscurl --service aoss --region us-east-1 -XGET \
    "https://my-collection-endpoint.us-east-1.aoss.amazonaws.com/_cat/indices"

Note: Replace us-east-1 with your AWS Region and the example endpoint with your endpoint.

Use verbose logging to confirm that the preceding headers are part of the request that's sent to the service. Incorrect signatures occur because of incorrect header values. You can also use verbose logging to confirm that your collection endpoint, Region, and service are correct.

Other "403" errors

Depending on the action that you perform, you might receive a different "403" error message.

For index specific ingestion operations, you might see an error message that's similar to the following:

"Authorization failure for the following indices: [index/collectionname/indexname]".

For search requests, you might see an error message such as "all shards failed" or "Bad Authorization".

For non-index specific operations, you might see an error message that's similar to the following:

"User does not have permissions for the requested resource".

To resolve these errors, make sure that the collection's data access policy includes the correct permissions for the collection and index resources. Also, make sure that the policy contains the correct IAM identity in the Principal section.

For more information on data access policy syntax, see Policy syntax.

"401 Unauthorized" errors

A "401" error usually occurs when the network policy denies access to either the OpenSearch Serverless APIs or the dashboard.

When you turn on verbose logging on your client or view the Network tab of the browser's developer console, you might get the following error:

"x-aoss-response-hint': 'X01:network-policy-deny".

To access OpenSearch Serverless APIs and the dashboard from the public internet, turn on public access in the network policy.

Example network policy in JSON format:

[
  {
    "Rules": [
      {
        "Resource": [
          "collection/collectionname"
        ],
        "ResourceType": "dashboard"
      },
      {
        "Resource": [
          "collection/collectionname"
        ],
        "ResourceType": "collection"
      }
    ],
    "AllowFromPublic": true
  }
]

Note: Replace collectionname with your collection's name.

For private access to the APIs and dashboard, create a VPC endpoint in the client VPC for OpenSearch Serverless.

Example network policy in JSON format:

[
  {
    "Rules": [
      {
        "Resource": [
          "collection/collectionname"
        ],
        "ResourceType": "dashboard"
      },
      {
        "Resource": [
          "collection/collectionname"
        ],
        "ResourceType": "collection"
      }
    ],
    "AllowFromPublic": false,
    "SourceVPCEs": [
      "vpce-0c9xxxxxxxxxxa680"
    ]
  }
]

Note: Replace collectionname with your collections name.

For other examples of network policies, see Network access for Amazon OpenSearch Serverless.

To access any OpenSearch Serverless collection from another VPC, create an OpenSearch Serverless VPC endpoint in the VPC. For more information, see Data plane access through AWS PrivateLink.

Note: You can choose to have your clients call the OpenSearch Serverless API endpoints from a VPC. You can access the dashboards from browsers that use the public internet.

To privately access the dashboards from your corporate network, use an inbound DNS resolver endpoint. The DNS query returns the correct private IP address. For more information, see Access Amazon OpenSearch Serverless collections using a VPC endpoint.

Related information

How do I troubleshoot access to OpenSearch Serverless Dashboards to view my collection?