how to run processing jobs in ec2 instances from a CI/CD set up in gitlab?

Question

I'm setting up a gitlab project, such that I can start a processing job ( for some data processing ) in an ec2 instance, this is an initial set up , but i'm planning to run some training for my regression models in these instance . for this, i have to store my aws user credentials in gitlab. is there a recommendentaion from aws ? or any articles/blogs/examples for such set up?

Answer

Storing AWS user credentials in GitLab is not recommended because it poses a security risk. Instead, you can use AWS IAM roles to grant access to your EC2 instances.

Here are some resources that can help you set up your GitLab project to run processing jobs in EC2 instances using IAM roles:

GitLab Runner documentation: https://docs.gitlab.com/runner/
AWS IAM Roles for EC2 documentation: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html
Blog post on running GitLab CI/CD on AWS: https://aws.amazon.com/blogs/opensource/running-gitlab-ci-cd-on-aws/
Example GitLab project with EC2 instance and IAM role setup: https://gitlab.com/alexta/awsgitlabcicd

how to run processing jobs in ec2 instances from a CI/CD set up in gitlab?

Relevant content