What's the best way to architecte ECS Task Definitions to enable re-use?

Question

I'm new to ECS (and containers in general) and need some help with architecture.  I'm have a simple Rails application and want to know the best practice for running it under ECS.  Also, I'm using CloudFormation for this configuration (not that it matters).

I have 2 containers; MyApp (a Rails application), Nginx (webserver).  These 2 containers are bundled into a single ECS TaskDefinition called "MyApp" under a service called "MyApp". 
 These tasks communicate with each other using ECS Service Connect over a "awsvpc" connector.  So far, so good.

However, I need to use the "MyApp" image for some additional things, not just serving web pages.  These things include daily tasks (backups, etc.), job processing, etc.

I could just create new task definitions for each task but that seems wasteful and unnecessary, especially given that the MyApp application requires a bunch of environment variables and take up a lot of room in my CFN template.

I could also create a new "Job" ECS Service and re-use the original "MyApp" task definition.  But that task definition also contains a webserver image (Nginx) that I don't need for this task.

What's the best way forward here?  Am I doing this wrong by combining an application and a webserver into a single ECS Task Definition?  Does each container image need it's own ECS Task Definition?  Do you then combine those tasks into a service?  If so, how do you configure them to communicate?

Accepted Answer

After looking over the ECS docs (again) I saw that they emphasized having only a single container for each task.  There are exceptions, such as a sidecar that directly modifies the "main" container, but as a general rule you don't want multiple containers in a task.  This helped me to understand that packaging the Nginx and MyApp containers in a single task was the wrong approach.

So, I reworked the system to only have a single container in each task; 1 task for Nginx, and 1 task for MyApp.  This works much better and it didn't change the networking at all.  In fact, my Nginx task is now much smaller and my MyApp container can be expanded a bit to process more threads.

Finally, I now have a one-to-one relationship between containers and tasks and I can spin up a new one-off task to do my processing (backups, db:migration, etc.) without having to worry about dragging an Nginx container along with me.

Answer

It is an interesting question.  Personally, using something like AWS Copilot CLI or CDK, I tend not to worry as much about reusing task definitions or environment variables taking up realestate as much as re-using the IaC code.  Using AWS CDK or AWS Copilot makes reusing the environment variables trivial.   That being said, I suppose you could create a 3rd sidecar/container that runs the same image but instead consumes an SQS queue triggered by your batch job to run your scripts and things like that.

Answer

Hello,

From what I understood from your question description since you are using 2 containers of your Rails-App and Nginx I assume they are also using two different separate images, if yes then you have separate services and tasks for both of them that way you can use your "My-App" image for various other things you mentioned.
Regarding the connectivity, I think u can go forward with the same method you are using right now to connect them or could go ahead with service discovery.

Let me know if you need more help

Thanks

What's the best way to architecte ECS Task Definitions to enable re-use?

Relevant content