How do I roll back a CodePipeline deploy action if a test action fails?

Question

How do I rollback my `RestSvc.Deploy` action to a previously deployed artifact (a Cfn template in this case), if my `E2ETests` action fails? I have a pipeline defined in CDK:

![A screenshot of a Beta CDK pipeline stage with 3 actions, RestSvc.Prepare, RestSvc.Deploy, and E2ETests](/media/postImages/original/IMN95htovXQl2V_rgp7_vvCg)

I have defined the pipeline in TypeScript:

```typescript
        const betaStack = new WeatherServiceStack(this, 'Beta', {
            env: {
                account: '975049914859',
                region: 'us-west-2',
            },
        })
        const betaStage = pipeline.addStage(betaStack);

betaStage.addPost(new pipelines.CodeBuildStep("E2ETests", {
            envFromCfnOutputs: {
                "WEATHER_API_URL": betaStack.apiUrl,
            },
            commands: [
                'python3 test_api.py'
            ]
        }));
```

I've searched the docs [high](https://docs.aws.amazon.com/codepipeline/latest/userguide/welcome.html) and [low](https://github.com/aws/aws-cdk/blob/v1-main/packages/@aws-cdk/pipelines/README.md) and I can't see anything. I also couldn't see it in the Console. Some folks on Google mention I might be able to trigger a rollback via CodeDeploy, but this pipeline uses the `LambdaRestApi` construct, which uses CloudFormation to deploy the changes to the Lambda code. CloudFormation can monitor CloudWatch Alarms, but I want to rollback if my _tests_ fail. Sometimes you have tests which cover important-but-less-used features that an alarm might not catch.

I'm happy to move my `E2ETests` action around if it helps.

Bonus points if you can tell me how to manually roll-back a given stage/environment (e.g. `Beta` or `Prod-YUL`) to a given build, without necessarily waiting for the whole pipeline to run (and maybe create different build artifacts -- especially dangerous if a dependency version update caused the outage!). Sometimes things slip by deployment alarms and tests, so the on-call has the job of pushing the rollback button 😅

Answer

To rollback the deployment if the E2E tests fail, you can add a rollback step after the E2ETests step in your pipeline stage.

The rollback step would use the AWS CLI 
cloudformation rollback
 command to roll back the CloudFormation stack deployed by the 
RestSvc.Deploy
 action.

For example:

Add a new CodeBuild step after E2ETests:

```

betaStage.addPost(new pipelines.CodeBuildStep(E2ETests, {
  //...
}))

betaStage.addPost(new pipelines.CodeBuildStep(Rollback, {
  commands: [
    'aws cloudformation rollback --stack-name RestSvcStack' 
  ]
}))
```

This will roll back the RestSvcStack if E2ETests fails. The pipeline will then continue to the next stage with the previous version deployed.

You may also want to add error handling or conditions to only execute the rollback step if E2ETests fails/errors out.

How do I roll back a CodePipeline deploy action if a test action fails?

相关内容