OpenSearch Ingestion Pipeline for S3 logs stops working after an hour

0

Hello,

I have created an OpenSearch Ingestion Pipeline for getting ALB Logs from S3 bucket, and pushing them to OpenSearch domain. The S3 source is setup with SQS queue.

When stopping/starting the pipeline it works fine and logs appear in OpenSearch domain. However after an hour or so ingestion stops. Looking at pipeline logs in CloudWatch I can see multiple errors from S3 like: Error processing from S3: Unable to execute HTTP request: Timeout waiting for connection from pool. Retrying with exponential backoff.

If I stop/start the pipeline it will again work for sometime and then throw these errors again. How can I solve this issue? Thanks

没有答案

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则