By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Database Schema Migration in CICD

0

Hello,

I am trying to do automatic database schema migration in CICD. My environment is ECS Fargate mode with RDS (Postgres), deployed with Gitlab and Terraform. I have some idea about the implementation and want to get some feedback on it.

The database schema migration files live in the same repo as the app. think something like django and it's db migration. For every code change, database schema migration will run first then the app deployment. But app deployment will only run if schema migration is a success. Here is more in detail:

Upon repository update, a new docker image is generated and pushed to ECR. This docker image has entrypoint of running database schema migration command. Then a terraform apply runs and deploy ECS task with this image which run the schema migration.

App deployment is triggered next. It will first check if the previous migration succeed and continue deploy if it's a success.

Since schema migration and app deployment are done in two steps, one way to record the migration result is to use SNS. I will need to implement extra SNS logic in the migration script. and the app deployment. But I don't like that coupling.

Is there better way?

Topics
Migration & ModernizationAnalyticsDatabaseServerlessContainers
Tags
Amazon Relational Database ServiceAWS Fargate
Language
English
James L
asked 5 months ago313 views
1 Answer
0

Hi James.

I had some experience with a similar process to handle the migrations on RDS (Postgres). We use GitHub, GitHub Actions, and CDK. So it's almost similar.

Every new change in the repo will run a GitHub Action that will trigger a cdk deploy and this will launch a Step-function that has a logic to validate if any change in the schema Is needed. This is done by triggering a container on ECS to run the migrations. If the execution goes well It will launch the app deployment.

Hope this gives you another idea to find the best workaround for you.

Jesus Castellanos
answered 5 months ago
  • James L
    5 months ago

    It seems you are doing two steps as well. Not sure if cdk works the same as terraform in this use case. How do you tell if the migration run successfully?

  • Jesus Castellanos
    5 months ago

    Hi James!

    I will copy the high-level logic from another answer which you can take a look at later:

    Step Functions invokes ECS/Fargate with a task token Step function then pauses waiting for that token to be returned. ECS/Fargate sends back a SendTaskScucess with that same task token. You can use the code mentioned above as an example to scaffold a solution from. You can see from the code the following: const params = { output: ""Callback task completed successfully."", taskToken: taskToken }; where can modify “output” to have any type of message you’d like. They use "Callback task completed successfully." but you can change this to be whatever you like such as a location of where data is stored in S3/DynamoDB/elsewhere.

    When the task token is received the workflow continues.

    Here is the original question: https://repost.aws/questions/QUMyCGFjUwSqaGSlUk99J1fg/aws-step-function-output-for-container-services

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content