Codebuild checkout of git modules fails with generic message

0

My repo has git submodules with one submodule has another level submodule

during the DOWNLOAD_SOURCE i receive a generic error like:

Submodule error authentication required for primary source and source version refs/heads/master
Submodule error authentication required for primary source and source version refs/heads/master 

this gives me no detail on which submodule caused the error. Is there more logging available ?

clogwog
asked 2 years ago3460 views
4 Answers
0
Accepted Answer

ended up putting the user:password of the .gitmodule in secretsmanager and then just before getting the submodules replacing git@ with user:password@ it works, just a bit disappointed in CodeBuild not supporting git modules out of the box. so the buildspec.yml file looks something like:

version: 0.2

env:
    git-credential-helper: yes
    secrets-manager:
        BB_USER_PASSWORD: "BitbucketAppUserPassword:BitbucketAppUserPassword"

phases:
    pre_build:
        commands:
            - echo replacing git credentials with BB_USER_PASSWORD in the shape of  user:password
            - sed -i "s/git@/${BB_USER_PASSWORD}@/g" .gitmodules
            - git submodule update --init --recursive
            - git submodule update --remote --merge
.....
  build:
        commands:
            - echo Build started on `date`
            - ./build2.sh
.....

remember to add a policy to the codebuild role to allow access the secrets manager key:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "secretsmanager:GetResourcePolicy",
                "secretsmanager:GetSecretValue",
                "secretsmanager:DescribeSecret",
                "secretsmanager:ListSecretVersionIds"
            ],
            "Resource": "<INSERT ARN of BitbucketAppUserPassword here>"
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": "secretsmanager:GetRandomPassword",
            "Resource": "*"
        }
    ]
}
clogwog
answered 2 years ago
0

Unfortunately, I don't think there is currently a way to get further details from the DOWNLOAD_SOURCE stage in CodeBuild.

That said, you might want to consider using CodePipeline here. With CodePipeline, the source stage can be configured as a full clone, which will then be passed into a build stage running CodeBuild. From here, you should be able to issue the needed git submodule commands in the install section in your buildspec.yml file.

The advantage of this is that since you are then moving the submodule init process into the build phases of your project, you should have full console output in the build logs, and you also have a bit more control over how the submodules are pulled. In doing this, you may need to enable git-credential-helper (1).

(1) https://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html#build-spec.env.git-credential-helper

AWS
SUPPORT ENGINEER
Wayne_G
answered 2 years ago
0

Thank you for that @Wayne_G I have a feeling that CodeBuild source doesn't support submodules at all.. Is there a know bug ?

I have been able to get it working late last night by

  1. switching of the submodule option in the CodeBuild source and specifying 1 level
  2. specifying the user/password inside the .gitmodule file for each git module
  3. in the build stage calling :
git submodule update --init --recursive
git submodule update --remote --merge

This, of coarse, is less than optimal for security purposes.

I've seen one other article where someone has this issue where they have switched to CodePipelines, but if this is a limitation of CodeBuild then at least remove the option to indicate that git modules are supported or allow the CodeStart connections to be used directly in CodeBuild.

clogwog
answered 2 years ago
0

I have seen this WooCommerce Custom Checkout Fields Plugin https://woocommerce.com/products/conditional-checkout-fields-for-woocommerce/

answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions