AWS lambda not triggering for all files upload to s3 bucket for Puppeteer based pdf generation in node

0

I have a pdf generation service using Puppeteer in AWS lambda, I have two s3 buckets, one will trigger the lambda when a JSON is uploaded into it and the resulting pdf is put into another s3 bucket.

S3(trigger) ---> Lambda ---> S3(result)

when uploading a small number of files (less than 100 or 50), it's working well, all files from the s3 trigger will get converted to pdf.

But when say 1000 files are uploaded, some results from the s3 bucket are missing, this will happen for less than 500 also. These missing files occur in a random fashion. Each request takes a maximum of 40 seconds to convert one JSON file to a pdf file

질문됨 7달 전233회 조회
1개 답변
0

Hello.

AWS Lambda has concurrency limits, both for the total number of concurrent executions and also for a given function. If you rapidly upload many files to S3, this could trigger a large number of concurrent Lambda executions, which may hit the concurrency limit.

Regards, Andrii

profile picture
전문가
답변함 7달 전
  • from what I have heard, there is a soft limit of 1,000 executions per account per region, so if I increase that, will it solve the issue? and this happens if I upload just 500 files.

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠