OpenSearch Create Repository

Hi,

Maybe it's not very obvious from the documentation, but I feel the repository cannot be created from the DevTools interface of Opensearch. I infer this from the statement "Even if you use HTTP basic authentication for all other purposes, you need to map the manage_snapshots role to your IAM user or role that has iam:PassRole permissions to pass TheSnapshotRole" in the Ref documentation

Using this Reference Document, try the following:

• For Permissions section in PreReq, assign the policy to a user which has AWS credentials or create a new one
• Add this user's ARN in the Mapping user sction
• Using those credentials and the python script provided in the document, you can create the repository from CLI ( basically this command is issued from a source external to Opensearch ). This source needs to have access to the Opensearch endpoint
• The subsequent commands to check the snapshot repository, status, initiate the snapshot can be triggered from the DevTools option of Opensearch

I was able to create the S3 snapshot repository with the above approach.

--Syd

Hello, thanks very much for your reply. I understand I can map an iam role as a backend role and I did try this. I don't understand how the trust portion would work correctly with a basic authenticated user. My trust policy was for opensearch. Do I need a trust policy which is open to principle * restricted by ip like the access policy? I'm not sure if that's even possible.

Hi,

Hope I understand your query correctly. I'll try to explain this in the context managing snaphots itself. You dont need an open to all (*) Trust policy. The trust policy for opensearch as already configured as per documentation ensures that it is able to connect to another AWS service, in this case S3. The basic auth user created during the domain creation is an admin user with full cluster management permissions from within the Opensearch interface. Both of these combine to ensure that you can create, delete, check snapshots. That essentially means there are two levels of permissions first Opensearch user + role and then AWS IAM role + Trust Policy

If you create a new basic auth user and assign it the some other role in opensearch - let's say readall_and_monitor, it wont be able to take snaphots though the trust policy allows opensearch to write to S3 buckets. It didnt even reach that point of asuming AWS IAM role since the role in Opensearch has prevented it from doing do.

If your cluster allows the basic auth user to do something, then the request would go the next level where Opensearch services assumes the IAM role and is able to write to S3

--Syd

Ok I'm still having issues with this. Here is exactly what I have.

1. Basic Auth user with a mapping to Opensearch Manage Snapshots Role.
2. Basic Auth user with permissions to backend iam role arn:aws:iam::REDACT:role/REDACT .
3. A IAM Role Trust Policy of

{
    "Version": "2008-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "es.amazonaws.com"
            },
            "Action": "sts:AssumeRole",
            "Condition": {
                "StringEquals": {
                    "aws:SourceAccount": "REDACT"
                },
                "ArnLike": {
                    "aws:SourceArn": "arn:aws:es:us-west-2:REDACT:domain/REDACT"
                }
            }
        }
    ]
}

4. The policy for this role is

{
    "Statement": [
        {
            "Action": [
                "s3:ListBucket"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::REDACT"
            ]
        },
        {
            "Action": [
                "s3:GetObject",
                "s3:PutObject"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::REDACT/*"
            ]
        },
        {
            "Action": "es:ESHttpPut",
            "Effect": "Allow",
            "Resource": "arn:aws:es:us-west-2:REDACT:domain/REDACT/*"
        },
        {
            "Action": [
                "iam:PassRole",
                "iam:GetRole"
            ],
            "Effect": "Allow",
            "Resource": "arn:aws:iam::REDACT:role/REDACT"
        }
    ],
    "Version": "2012-10-17"
}

5. HTTPS Post Payload of

PUT _snapshot/manual_snapshot
{
  "type": "s3", 
  "settings": 
  {
    "bucket": "REDACT", 
    "base_path": "REDACT", 
    "server_side_encryption": "true",
    "canned_acl": "private", 
    "storage_class": "standard", 
    "region": "us-west-2", 
    "role_arn": "arn:aws:iam::REDACT:role/REDACT",     
    "readonly": "true"
  }
}

6. I am still getting when making this post via DevTools as the proper Basic Auth User and also making the post outside of DevTools.

{"Message":"User: anonymous is not authorized to perform: iam:PassRole on resource: arn:aws:iam::REDACT:role/REDACT because no resource-based policy allows the iam:PassRole action"}

1. Basic Auth user with a mapping to Opensearch Manage Snapshots Role. This is correct but this user can only be used for managing snapshots. It cannot be used for creating the snapshot repository. Assuming username is esuser1

Also create an AWS IAM user and provide that user's ARN in the list of mapped users for manage_snapshot role in opensearch. So that would mean you have two users mapped. One internal user in elasticsearch and another AWS IAM user with it's ARN specified.

2. Basic Auth user with permissions to backend iam role arn:aws:iam::REDACT:role/REDACT This wont work. The backend roles in opensearch are used to map to external entities like LDAP / Active Directory.

3. A IAM Role Trust Policy of (assume policy name is SnapshotRole) This is fine

4. The policy for this role is (assume policy name is SnapshotRole) Remove these permissions from the policy and assign it to the AWS IAM user created in Step 1. Create Access Key and Secret key for the IAM user.

{
            "Action": "es:ESHttpPut",
            "Effect": "Allow",
            "Resource": "arn:aws:es:us-west-2:REDACT:domain/REDACT/*"
        },
        {
            "Action": [
                "iam:PassRole",
                "iam:GetRole"
            ],
            "Effect": "Allow",
            "Resource": "arn:aws:iam::REDACT:role/REDACT"
        }

Apart from the above other permissions are fine. The name you assign to the role (ie. SnapshotRole) is what you substitute in "arn:aws:iam::REDACT:role/REDACT" when creating permission for IAM user.

5. HTTPS Post Payload of Refer to Step 1 and 2 in https://docs.aws.amazon.com/opensearch-service/latest/developerguide/managedomains-snapshots.html#managedomains-snapshot-registerdirectory. Step 1 is something you would have done earlier as part of manage_snapshot mapping but cross check at this point The process to create the repo is external to Dev Tools. Try using the python script in the above reference document and substitute the required variables and parameters. Here you need to use the AWS IAM user that you created not the internal user.

Once the above repository registration is successful you can take snapshots by logging as the opensearch user you created. ie. esuser1 as assumed earlier

Summarizing: Basic auth internal user is used to create snapshots. AWS IAM user is used to create snapshot respository (required only once for repository creation).

--Syd