Loading streaming data from Dynamodb into ElasticSearch - {"error":"no handler found for uri [//lambda-index/_doc/CUSTOMER] and method [PUT]"}


Loading streaming data from dynamodb stream to ElasticSearch via lambda is described here. I am facing following errors when triggering lambda for test event.

{"error":"no handler found for uri [//lambda-index/_doc/CUSTOMER] and method [PUT]"}
{"error":"no handler found for uri [//lambda-index/_doc/CUSTOMER] and method [DELETE]"}

Checked the same api calls at OpenSearch Dashboard Console and apis work fine:

PUT /lambda-index/_doc/CUSTOMER#34567
DELETE /lambda-index/_doc/CUSTOMER#34567

This suggests me that the error is somehow in formatting URL while making call via request() module in Python. Also the character '#' (inside CUSTOMER#34567) in the api call gets truncated (CUSTOMER#12345 and CUSTOMER#987654 are both taken up as CUSTOMER when they should be different and as given).

Do you have an idea to correct the mistake here.

Lambda code:

import boto3
import requests
from requests_aws4auth import AWS4Auth

region = 'ap-south-1' # e.g. us-east-1
service = 'es'
credentials = boto3.Session().get_credentials()
awsauth = AWS4Auth(credentials.access_key, credentials.secret_key, region, service, session_token=credentials.token)

host = 'https://vpc-elasticsearch-temp-au5shucd5nkecivdx7cnm7jaqi.ap-south-1.es.amazonaws.com/' 
index = 'lambda-index'
type = '_doc'
url = host + '/' + index + '/' + type + '/'

headers = { "Content-Type": "application/json" }

def handler(event, context):
    count = 0
    for record in event['Records']:
        # Get the primary key for use as the OpenSearch ID
        id = record['dynamodb']['Keys']['PK']['S']

        if record['eventName'] == 'REMOVE':
            r = requests.delete(url + id, auth=awsauth)
            document = record['dynamodb']['NewImage']
            r = requests.put(url + id, auth=awsauth, json=document, headers=headers)
        count += 1
    return str(count) + ' records processed.'

Versions: Python 3.9 requests v 2.27.1 ElasticSearch: OpenSearch1.2

gefragt vor 2 Jahren2290 Aufrufe
1 Antwort

I believe this is happening because you have a trailing slash in the host field. Other people have reported this same behavior: https://github.com/elastic/elasticsearch-php/issues/1014

Can you try removing the trailing slash? i.e. https://vpc-elasticsearch-temp-au5shucd5nkecivdx7cnm7jaqi.ap-south-1.es.amazonaws.com

profile pictureAWS
beantwortet vor 2 Jahren

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen