Codebuild suddenly fails in DOWNLOAD_SOURCE phase

0

Hello,
Starting today our codebuild project, which has been working flawlessly for months and not modified has stopped working.
Failure occurs in DOWNLOAD_SOURCE phase, with this message: "SINGLE_BUILD_CONTAINER_DEAD: Build container found dead before completing the build. Build container died because it was out of memory, or the Docker image is not supported."
The applications code is hosted on CodeCommit.
We did not do any modification to the buildspec file, and the very same build was even still working yesterday.

We are using aws/codebuild/amazonlinux2-x86_64-standard:2.0 images, and did not do any modifications but the failure suddenly happened today. During the previous build the job was using at most 20% of the available memory (15GB)
Sadly nothing gets logged because it fails even before processing the first commands of the buildspec so it's nearly impossible to debug.

Do you guys have any ideas on what could cause this ?

Thanks,
Daniel

Edited by: drolland on Oct 20, 2020 2:29 AM

Edited by: drolland on Oct 20, 2020 2:30 AM

질문됨 3년 전2969회 조회
2개 답변
1

So I finally found the reason by creating some test build, so I'm sharing the solution I found below.

Solution

If you are using multiple sources for your build project, get sure that Git Clone Depth is set to 1 for all your sources. <br>
Git clone depth defaults to "Full", so if you have built your project with CloudFormation get sure to include GitCloneDepth: 1 in your template SecondarySources list.

Explanation

Not sure about what has changed internally, but there seems to be some kind of timeout when cloning the repos, even though the error message is unclear about that. <br> As of today (2020-10-20), this error is reproductible if you have multiple repos containing enough commits, by creating an empty test project and including 3 additional sources. The build will fail with the same error message before starting.

Once I was able to reproduce the issue, I've tried fiddling around with clone depth and it finally worked. Again, I'm not sure if this is temporary or will be fixed, but as of yesterday (2020-10-19), this workaround was not necessary.

Edited by: drolland on Oct 20, 2020 5:37 AM

답변함 3년 전
  • You are a lifesaver mate. It indeed was a git clone depth issue. When I set it to 1, the code build job started working again. In my case, even the logs were not getting printed in the cloudwatch. Thanks a ton!

0

Hi drolland,

Would you PM me build ARNs of affected builds? Preferably one before and one after you noticed this behavior change. I'd like to help investigate this further.

답변함 3년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠