CodePipeline - how to pass and consume multiple artifacts across CodeBuild Steps?

Question

Hello.  I know I can pull multiple Source actions (e.g. 2 GitHub repos) into a CodeBuild step by adding multiple Source Artifacts. It would seem that i should follow the same pattern (e.g. $CODEBUILD_SRC_DIR__) where individual CodeBuild actions are producing and consuming artifacts. In my case, my consuming step needs the following inputs:

1. Source input (primary source, to get my buildspec.yml) 
2. The output of a prior CodeBuild step (a couple of files).

I am successful in setting up the Input/Output artifacts in my pipeline and validating that the zipped files are placed into the pipeline s3 bucket. Its all good up until that point.

However, when I run the consuming step and troll the file system I do not see anything like $CODEBUILD_SRC_DIR__.

Is this scenario supported? if so, how do I reference the secondary artifacts in my CodeBuild buildspec?

Thanks Dave

Answer

It is supported and the name you're looking for is correct. So I think you just have a small error.

Here's an example (CDK Typescript): https://github.com/awslabs/aws-greengrass-labs-component-for-the-things-stack-lorawan/blob/main/cicd/lib/cicd-stack.ts

The output from the build stage is passed as an extra input to the deploy stage. Both of which are CodeBuild actions:

```
        const buildAction = new codepipeline_actions.CodeBuildAction({
            actionName: `${Names.PREFIX_DASH}-build`,
            project: buildProject,
            input: source,
            outputs: [build]
        });

const deployAction = new codepipeline_actions.CodeBuildAction({
            actionName: `${Names.PREFIX_DASH}-deploy`,
            project: deployProject,
            input: source,
            extraInputs: [build],
            outputs: [deploy],
            environmentVariables: {
                "GREENGRASS_CORE_NAME": { value: context.greengrassCoreName }
            }
        });
```

And the buildspec for the deploy stage, that consumes the output from the build stage: https://github.com/awslabs/aws-greengrass-labs-component-for-the-things-stack-lorawan/blob/main/cicd/deployspec.yaml

```      
      # Get the recipe file generated by the build stage
      - RECIPE_FILE=$(ls $CODEBUILD_SRC_DIR_Build/aws.greengrass.labs.*.json)
```

Answer

How about [caching in AWS CodeBuild](https://docs.aws.amazon.com/codebuild/latest/userguide/build-caching.html). A cache can store reusable pieces of your build environment and use them across multiple builds. Your build project can use one of two types of caching: Amazon S3 or local.

CodeBuild also supports secondaryArtifacts. These artifacts use the secondary-artifacts block of the buildspec file that is nested inside the artifacts block.

CodePipeline - how to pass and consume multiple artifacts across CodeBuild Steps?

Relevant content