Fargate timeout problem

1

Hi. I've got an unexpected error, I guess it's about timeout. Task takes time quite long. generally less than 1minute. There is no problem when it finish before 1 minute. But sometimes task takes more than 1 minute. Error occurs when it takes more than 1minute. Please see below.

[ec2-user@ip-000-00-0-00 ~]$ curl --location --request POST 'userid.ap-northeast-2.elb.amazonaws.com:port/service' -d "video.mp4" -o output.json -v
Note: Unnecessary use of -X or --request, POST is already inferred.
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 172.31.74.35:5000...
* Connected to loadbalancer.ap-northeast-2.elb.amazonaws.com (000.00.00.00) port 0000 (#0)
> POST /service HTTP/1.1
> Host: userid.ap-northeast-2.elb.amazonaws.com:port
> User-Agent: curl/7.79.1
> Accept: */*
> Content-Length: 37
> Content-Type: application/x-www-form-urlencoded
>
} [37 bytes data]
100    37    0     0    0    37      0      0 --:--:--  0:00:59 --:--:--     0* Mark bundle as not supporting multiuse
< HTTP/1.1 500 Internal Server Error
< Date: Tue, 19 Apr 2022 01:45:23 GMT
< Content-Type: application/octet-stream
< Content-Length: 0
< Connection: keep-alive
< Server: Python/3.7 aiohttp/3.7.4.post0
<
100    37    0     0    0    37      0      0 --:--:--  0:01:00 --:--:--     0
* Connection #0 to host loadbalancer.ap-northeast-2.elb.amazonaws.com left intactm left intact

With curl verbose option, I get 500 Internal Server Error at 0:00:59. How can I finish my task which takes more than 1minutes? I've tried

  • increasing Health check grace period for ECS Service
  • increasing idle timeout of Load Balancer
  • increasing timeout and interval of Target group
  • curl options (like keep-alive, max-time) My EC2 Instance
  • type : t2.micro
  • Amazon Linux My Service
  • Service type : REPLICA
  • Launch type : FARGATE My task in service
  • Network mode : awsvpc
  • Compatibilities : EC2, FARGATE
  • Requries compatibilities : FARGATE
  • EFS mounted
  • Docker Appreciate,
Shuan
asked 2 years ago4044 views
2 Answers
1

I know this is an old question, but I was having a similar problem and thought I would provide my experience in case it would help anyone else (for the record, I think the answer of "Make your application faster" is not helpful at all). In my case, we are hosting a website with Fargate and fronting it with CloudFront origins. Long running APIs would timeout and return a 504 to the client due to the CloudFront request timeout setting on the origin. By default, the max you can set this timeout is 60 seconds, but you can do a service limit request and have it bumped up to 180 seconds. That is their current hard limit and they won't go higher than that. Once I did that and updated my CloudFront origin, my 504 errors went away.

jkehoe
answered a year ago
-1

Request / Response taking longer than a minute is already in very bad standing.

Make your application faster or redesign it to be batched / async?

kai
answered 2 years ago
  • It's machine learning service so it takes long. We can't make our application faster than this.

  • Websockets is another legitimate use case where you don't want the connection to be automatically timed out. Anyone figure out where the timeout is set?

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions