API + Lambda has #503 error 20% of the time


Running a step function state machine accessing an AWS HttpApi to run a lambda using a docker container of ~3 GB size. Fails 20% of the time with #503 error, the rest of the time I get the exact results I expect. It's not a container warm up I believe. Logs for API and lambda (non-container and container) don't show any problems. I have maximized the step functions time out, the retries for the specific lambda, the memory for the lambda (10GB). I have also tried to ping the container and wait 30 seconds in the step functions to no avail.

*** Edit: Should have mentioned concurrency for the API is at max (believe it is automatic for HttpApis)

Any advice would be appreciated. I don't want to use ECS fargate due to cost and relative complexity. Thanks!

1 Answer
Accepted Answer

Although not a direct answer to your question, but have you considered using AppRunner? It runs your docker service and will hibernate if there are no requests saving you money. So very similar to the way lambdas work, but without the complexity if you are using it with docker. I've found it easy to setup and very useful. AWS AppRunner

AWS App Runner is a fully managed service that makes it easy for developers to quickly deploy containerized web applications and APIs, at scale and with no prior infrastructure experience required. Start with your source code or a container image. App Runner builds and deploys the web application automatically, load balances traffic with encryption, scales to meet your traffic needs, and makes it easy for your services to communicate with other AWS services and applications that run in a private Amazon VPC. With App Runner, rather than thinking about servers or scaling, you have more time to focus on your applications.

answered a year ago
  • Thanks for the mention! I will give it a look-see. I am using caddy in other parts of my workflows (and have used dokku in the past) so this should be a easy setup.

  • Glad to help :)

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions