How to correctly issue post commands to opensearch cluster in VPC?

0

Hi, I've created an opensearch cluster in a VPC. I'm able to "GET" the cluster details by hitting <vpc endpoint>/_cluster/settings but am not having luck when trying to "POST". Which URL do I need to POST to when the opensearch cluster is deployed in a VPC? Should it work with the <vpc endpoint>/_cluster/settings ?

My working GET command -

def lambda_handler(event, context):
    x = requests.get('https://<vpc endpoint>/_cluster/settings?include_defaults=true')

output - Function Logs

:"5s","max_index_buffer_size":"-1","shard_inactive_time":"5m","index_buffer_size":"10%","min_index_buffer_size":"48mb..."

POST command that is not working -

def lambda_handler(event, context):
    url = 'https://vpc-XXXXXX-us-east-1.es.amazonaws.com/_cluster/settings'
    myobj = {"SnapshotOptions": {
    "AutomatedSnapshotStartHour": 3
  } }
    x = requests.post(url, json = myobj)

output - Function Logs

START RequestId: b483f2ca-0051-468a-81cf-8a771a667bd2 Version: $LATEST
{"Message":"Your request: '/_cluster/settings' is not allowed for verb: POST"}

documentation I followed - https://docs.aws.amazon.com/opensearch-service/latest/developerguide/configuration-api.html#configuration-api-actions-describedomainconfig

1 Answer
0

Hi,

You need to use the UpdateDomainConfig API to set configure the Snapshot Options

eg: POST request to the below mentioned URL with the required payload

https://es.us-east-1.amazonaws.com/2021-01-01/opensearch/domain/<DomainName>/config

--Syd

profile picture
Syd
answered a year ago
  • When I try that, the request times out. I believe this might be because the opensearch cluster is deployed into a vpc and not publicly available. The only link that has worked for anything is the VPC endpoint. And yes, I am inside the VPC when attempting these requests.

  • I think @Syd, pointed out the API to use. the example should have been modified to use your vpc endpoint. https://vpc-XXXXXX-us-east-1.es.amazonaws.com/2021-01-01/opensearch/domain/<DomainName>/config

    can you try that?

  • Thanks, I tried the POST to https://<my vpc>-east-1.es.amazonaws.com/2021-01-01/opensearch/domain/<my_domain_name>/config but I received an error -

    {"error":"no handler found for uri [/2021-01-01/opensearch/domain/<my_domain_name>/config] and method [POST]"}

  • I created this script and tested from a node in Private subnet which can reach the internet via NAT

    import boto3
    client = boto3.client('opensearch')
    response = client.update_domain_config(
         DomainName='<domain_name>',
         SnapshotOptions={'AutomatedSnapshotStartHour': 12})
    

    When i removed the NAT connectivity it went unresposive (similar to the timeout you mentioned). So does Lambda script have outbound connectivity to the internet? Coz it needs access to es.us-east-1.amazonaws.com ie. opensearch endpoint.

  • The lambda does not have access to the internet. It's in the same vpc and subnet as the opensearch cluster. Neither have access to the internet. They're completely private. I have a VPC endpoint to use. Is allowing internet access the only option?

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions