OpenSearch Ingestion Pipeline for S3 logs stops working after an hour

0

Hello,

I have created an OpenSearch Ingestion Pipeline for getting ALB Logs from S3 bucket, and pushing them to OpenSearch domain. The S3 source is setup with SQS queue.

When stopping/starting the pipeline it works fine and logs appear in OpenSearch domain. However after an hour or so ingestion stops. Looking at pipeline logs in CloudWatch I can see multiple errors from S3 like: Error processing from S3: Unable to execute HTTP request: Timeout waiting for connection from pool. Retrying with exponential backoff.

If I stop/start the pipeline it will again work for sometime and then throw these errors again. How can I solve this issue? Thanks

답변 없음

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인