Segregation of EC2 instances in one VPC

Question

Hello. I'm running a very simple infrastructure where in one VPC I have two types of instances - "workers" that are constantly working on certain tasks and "monitors" that calculate various performance metrics of the work done by the workers. Monitors don't connect directly to the workers, they get the data from the database and also they can receive/send various control signals from/to the workers via Redis ElasticCache pub/sub. There are a thousand of workers and only about 10 monitors. As my project grew larger I want to give full access to the AWS Console to an employee who will manage the monitors but don't want them to be able to access or even see the workers in the console. I was thinking about creating two separate VPCs for this but from what I read this might not be the best option. What is the best way to do this?

Answer

Hello.

If the worker's EC2 and monitor's EC2 are in the same AWS account's VPC, if you give them full access rights (AdministratorAccess) to the AWS account, they will be able to access the worker's AWS resources.  
Therefore, we recommend that you create an IAM user ,IAM policy and IAM group by creating an IAM policy that can only operate the monitor's AWS resources, rather than giving full AWS access.  
By doing this, even if you can display the worker's AWS resources, you will not be able to edit or delete them, so you will only be able to perform operations according to your role.  
For example, as shown in the document below, it is possible to use IAM policies to manipulate resources with specific tags.  
By using this, I think it is possible to set it so that only the resources that match the tag set for the AWS resource of the monitor can be operated.  
https://repost.aws/knowledge-center/iam-tag-based-restriction-policies

Or, if you want to completely separate them, you can also separate AWS accounts.  
This document describes an example of separating the production environment and development environment, but I think you could also use it to separate the worker environment and monitor environment.  
https://docs.aws.amazon.com/prescriptive-guidance/latest/strategy-unlock-value-data-financial-services/best-practices-ml-ops.html  
https://docs.aws.amazon.com/organizations/latest/userguide/orgs_introduction.html  
> AWS best practices for account management recommend that you divide your accounts into four accounts for each use case: experimentation, dev, test, and prod. It’s also a best practice to have a governance account for providing shared MLOps resources across the organization and a data lake account for providing centralized data access. The rationale for this is to completely separate the development, test, and production environments, avoid delays caused by service limits being hit through multiple use cases and data science teams sharing the same set of accounts, and provide a complete overview of the costs for each use case. Finally, it’s a best practice to separate account-level data, as each use case has its own set of accounts.

Answer

And as Riku pointed out, with his second option, if latency is not a problem and workers and monitors teams need to follow segregation of duties principles, then account strategy may be preferred

Segregation of EC2 instances in one VPC

Relevant content