Loading streaming data from Dynamodb into ElasticSearch - {"error":"no handler found for uri [//lambda-index/_doc/CUSTOMER] and method [PUT]"}

0

Loading streaming data from dynamodb stream to ElasticSearch via lambda is described here. I am facing following errors when triggering lambda for test event.

{"error":"no handler found for uri [//lambda-index/_doc/CUSTOMER] and method [PUT]"}
{"error":"no handler found for uri [//lambda-index/_doc/CUSTOMER] and method [DELETE]"}

Checked the same api calls at OpenSearch Dashboard Console and apis work fine:

PUT /lambda-index/_doc/CUSTOMER#34567
DELETE /lambda-index/_doc/CUSTOMER#34567

This suggests me that the error is somehow in formatting URL while making call via request() module in Python. Also the character '#' (inside CUSTOMER#34567) in the api call gets truncated (CUSTOMER#12345 and CUSTOMER#987654 are both taken up as CUSTOMER when they should be different and as given).

Do you have an idea to correct the mistake here.

Lambda code:

import boto3
import requests
from requests_aws4auth import AWS4Auth

region = 'ap-south-1' # e.g. us-east-1
service = 'es'
credentials = boto3.Session().get_credentials()
awsauth = AWS4Auth(credentials.access_key, credentials.secret_key, region, service, session_token=credentials.token)

host = 'https://vpc-elasticsearch-temp-au5shucd5nkecivdx7cnm7jaqi.ap-south-1.es.amazonaws.com/' 
index = 'lambda-index'
type = '_doc'
url = host + '/' + index + '/' + type + '/'

headers = { "Content-Type": "application/json" }

def handler(event, context):
    count = 0
    for record in event['Records']:
        # Get the primary key for use as the OpenSearch ID
        id = record['dynamodb']['Keys']['PK']['S']

        if record['eventName'] == 'REMOVE':
            r = requests.delete(url + id, auth=awsauth)
        else:
            document = record['dynamodb']['NewImage']
            r = requests.put(url + id, auth=awsauth, json=document, headers=headers)
        count += 1
    return str(count) + ' records processed.'

Versions: Python 3.9 requests v 2.27.1 ElasticSearch: OpenSearch1.2

rejji
asked 2 years ago2190 views
1 Answer
1

I believe this is happening because you have a trailing slash in the host field. Other people have reported this same behavior: https://github.com/elastic/elasticsearch-php/issues/1014

Can you try removing the trailing slash? i.e. https://vpc-elasticsearch-temp-au5shucd5nkecivdx7cnm7jaqi.ap-south-1.es.amazonaws.com

profile pictureAWS
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions