- Neueste
- Die meisten Stimmen
- Die meisten Kommentare
Hello,
Thank you for providing all the details. From it, I could see this: (...) upload the file to S3 and after each upload I record a log message with file name. (...)
.
I'm assuming that your service doesn't have a Loadbalancer in front of it. Thus, the ECS will send the SIGTERM signal to your task and 30 seconds after (by default) it will send a SIGKILL command if the application doesn't stop by itself. It looks like that the process is being stopped before the log message be recorded.
I'd recommend you to check the stopTimeout configuration. It could be configured using either the Task definition parameters or ECS Agent configuration using ECS_CONTAINER_STOP_TIMEOUT
(for EC2 only). Fargate has a limit of 120 seconds and there is no limit for EC2.
In order to help you more, I'm sharing this blog post that has some code snippets for SIGTERM handling on different languages. You can use something similar to make sure that the S3 file is completely copied and logs are shipped before stop the process.
Talking more about CloudWatch Logging driver, it will use the blocking
mode by default. Which means that, whenever there is something to send to the CW Logs, it will stop the main process to send the logs. This guarantee that the logs will be delivered without miss them. You can also have the non-blocking
mode configured which will buffer the logs in the container internal memory and send to the endpoint whenever is possible (best effort basis). You can check which is the mode configured into your logConfiguration. Please, find the further information about it on this documentation.
Hope this helps you to solve the issue.
Relevanter Inhalt
- AWS OFFICIALAktualisiert vor 9 Monaten
- AWS OFFICIALAktualisiert vor 2 Jahren
- AWS OFFICIALAktualisiert vor 2 Jahren
- AWS OFFICIALAktualisiert vor 2 Jahren
Thanks for Your Response !! I would like to Add that -
It looks like the app is getting shutdown as soon as there is a SIGTERM signal. You can test it from a local environment sending the SIGTERM to the process and checking if it will be terminated immediately. In order to have the stopTimeout working, the application shouldn't terminate at SIGTERM signal. The stopTimeout is the time that the ECS Agent will wait to send the SIGKILL signal. If the application is terminating on SIGTERM, the stopTimeout is not triggered. I'd check the app signal handling to fix this issue.