OpenSearch Ingestion Pipeline for S3 logs stops working after an hour

0

Hello,

I have created an OpenSearch Ingestion Pipeline for getting ALB Logs from S3 bucket, and pushing them to OpenSearch domain. The S3 source is setup with SQS queue.

When stopping/starting the pipeline it works fine and logs appear in OpenSearch domain. However after an hour or so ingestion stops. Looking at pipeline logs in CloudWatch I can see multiple errors from S3 like: Error processing from S3: Unable to execute HTTP request: Timeout waiting for connection from pool. Retrying with exponential backoff.

If I stop/start the pipeline it will again work for sometime and then throw these errors again. How can I solve this issue? Thanks

Aucune réponse

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions