Codebuild suddenly fails in DOWNLOAD_SOURCE phase

0

Hello,
Starting today our codebuild project, which has been working flawlessly for months and not modified has stopped working.
Failure occurs in DOWNLOAD_SOURCE phase, with this message: "SINGLE_BUILD_CONTAINER_DEAD: Build container found dead before completing the build. Build container died because it was out of memory, or the Docker image is not supported."
The applications code is hosted on CodeCommit.
We did not do any modification to the buildspec file, and the very same build was even still working yesterday.

We are using aws/codebuild/amazonlinux2-x86_64-standard:2.0 images, and did not do any modifications but the failure suddenly happened today. During the previous build the job was using at most 20% of the available memory (15GB)
Sadly nothing gets logged because it fails even before processing the first commands of the buildspec so it's nearly impossible to debug.

Do you guys have any ideas on what could cause this ?

Thanks,
Daniel

Edited by: drolland on Oct 20, 2020 2:29 AM

Edited by: drolland on Oct 20, 2020 2:30 AM

asked 2 years ago743 views
2 Answers
1

So I finally found the reason by creating some test build, so I'm sharing the solution I found below.

Solution

If you are using multiple sources for your build project, get sure that Git Clone Depth is set to 1 for all your sources. <br>
Git clone depth defaults to "Full", so if you have built your project with CloudFormation get sure to include GitCloneDepth: 1 in your template SecondarySources list.

Explanation

Not sure about what has changed internally, but there seems to be some kind of timeout when cloning the repos, even though the error message is unclear about that. <br> As of today (2020-10-20), this error is reproductible if you have multiple repos containing enough commits, by creating an empty test project and including 3 additional sources. The build will fail with the same error message before starting.

Once I was able to reproduce the issue, I've tried fiddling around with clone depth and it finally worked. Again, I'm not sure if this is temporary or will be fixed, but as of yesterday (2020-10-19), this workaround was not necessary.

Edited by: drolland on Oct 20, 2020 5:37 AM

answered 2 years ago
  • You are a lifesaver mate. It indeed was a git clone depth issue. When I set it to 1, the code build job started working again. In my case, even the logs were not getting printed in the cloudwatch. Thanks a ton!

0

Hi drolland,

Would you PM me build ARNs of affected builds? Preferably one before and one after you noticed this behavior change. I'd like to help investigate this further.

answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions