Serverless vs EKS/EC2
Dear Team,
We are building a service which will be exposed as Rest API through API Gateway. The service will perform 3 things.
- Validate the inputs
- Upload a document to S3(rather move the document-approx 40kb in size from Staging S3 location to Target S3)
- Insert the metadata to Dynamo DB
We expect this service to be called approx 2 lakh times within 8-10 hours on daily basis through AWS Batch. Afterwards, it may not be used through out the day. We were planning to have a lambda function for each of the above steps which will be called from API Gateway. If required, we may use Step functions to orchestrate above 3 steps.
Would Lambda functions be the appropriate way to process these steps in the service or should we use EC2/EKS and ASG to scale in and Scale out.
I assume the service will respond back within second but bearing Lambda concurrency limits in mind and any other limits we may face, let me know the best way to process the batch.
Regards, Dhaval Mehta
Yes, using Lambda here, see documentation - https://docs.aws.amazon.com/lambda/latest/dg/services-s3-batch.html
There is also a blog post on batch processing - https://aws.amazon.com/blogs/compute/creating-aws-serverless-batch-processing-architectures/
And some best practices - https://docs.aws.amazon.com/lambda/latest/dg/best-practices.html
If you build your Lambda function with containers, then you can consider running the containers in EKS in future should your use case change.
Yes; the invocation mechanisms will be different. The core remains the same though.
Relevant questions
what are some ways/alternative to expose sagemaker endpoints as a HTTP /REST endpoints?
asked 3 months agoServerless vs EKS/EC2
Accepted Answerasked 5 months agoUploading greater than 6 MB file to S3 through API Gateway results in Request too long error, Is that expected ?
asked 2 months agoAWS Api Gateway: how to expose new services automatically?
asked 3 months agoHow to use postgres rest api in AWS
asked 5 days agoApi Gateway Payload size limit increase
asked 5 years agoIs it possible to send json data in body of GET method defined in API Gateway REST API ?
Accepted Answerasked 2 months agoFor S3 Intelligent Tiering which exact API operations are required for an object to be marked as `Accessed`?
asked 2 months agoBest practices for securing service to service REST communication in a microservices architecture
Accepted Answerasked 2 years agoLooking for a best-practice for building serverless application with amplify and api gateway and lambda
Accepted Answer
You can't really just use the same container image and run it on a traditional container solution (ECS/EKS/Docker/etc). Lambda has a specific event driven programming model that do not exist on the other platforms. The container image is just a packaging mechanism (what runs inside the image is different between Lambda and the other platforms)