Skip to content
By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post
About re:Post

HTTP 502 while running a CodeBuild project

0

Hello, I've created a CodeBuild project in which buildspec.yamlfile I'm compiling, building and running unit and integration tests. Here is the file:

version: 0.2
phases:
  install:
    commands:
      - echo "build started by ${CODEBUILD_INITIATOR}"
      - echo "running on:"
      - java -version
      - mvn -version
  build:
    commands:
      - echo "*** Compiling and running unit tests"
      - ./run-local.sh
      - echo "*** Performing integration tests"
      - ./deploy.sh
      - echo "*** Cleaning up work"
      - ./cleanup.sh

Running the CodeBuild project fails in the maven's unit tests phase of the CodeBuild build phase, while trying to run the file run-local.sh , with the exception stack below:

[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running fr.simplex_software.quarkus_aws.tests.BedrockAiResourceIT
2024-05-09 16:37:23,250 INFO  [io.qua.ama.lam.run.MockEventServer] (build-2) Mock Lambda Event Server Started
Invoking io.quarkus.amazon.lambda.runtime.QuarkusStreamHandler::handleRequest (java17)
Decompressing /codebuild/output/src1806848931/src/git-codecommit.eu-west-3.amazonaws.com/v1/repos/bedrock-gateway/bedrock-gateway-api/target/function.zip
Local image is up-to-date
Using local image: public.ecr.aws/lambda/java:17-rapid-x86_64.
Mounting /tmp/tmp73nu0m4a as /var/task:ro,delegated, inside runtime container
Invoking io.quarkus.amazon.lambda.runtime.QuarkusStreamHandler::handleRequest (java17)
Decompressing /codebuild/output/src1806848931/src/git-codecommit.eu-west-3.amazonaws.com/v1/repos/bedrock-gateway/bedrock-gateway-api/target/function.zip
Using local image: public.ecr.aws/lambda/java:17-rapid-x86_64.
Mounting /tmp/tmp3zz3dzkx as /var/task:ro,delegated, inside runtime container
Invoking io.quarkus.amazon.lambda.runtime.QuarkusStreamHandler::handleRequest (java17)
Decompressing /codebuild/output/src1806848931/src/git-codecommit.eu-west-3.amazonaws.com/v1/repos/bedrock-gateway/bedrock-gateway-api/target/function.zip
Using local image: public.ecr.aws/lambda/java:17-rapid-x86_64.
Mounting /tmp/tmpz9xu8hhd as /var/task:ro,delegated, inside runtime container
Timed out while attempting to establish a connection to the container. You can increase this timeout by setting the SAM_CLI_CONTAINER_CONNECTION_TIMEOUT environment variable. The current timeout is 60.0 (seconds).
Invalid lambda response received: Lambda response must be valid json
2024-05-09 16:38:28 127.0.0.1 - - [09/May/2024 16:38:28] "POST /bedrock/mistral2 HTTP/1.1" 502 -
Invoking io.quarkus.amazon.lambda.runtime.QuarkusStreamHandler::handleRequest (java17)
Decompressing /codebuild/output/src1806848931/src/git-codecommit.eu-west-3.amazonaws.com/v1/repos/bedrock-gateway/bedrock-gateway-api/target/function.zip
Using local image: public.ecr.aws/lambda/java:17-rapid-x86_64.
Mounting /tmp/tmp4tl7e9ao as /var/task:ro,delegated, inside runtime container
Timed out while attempting to establish a connection to the container. You can increase this timeout by setting the SAM_CLI_CONTAINER_CONNECTION_TIMEOUT environment variable. The current timeout is 60.0 (seconds).
Invalid lambda response received: Lambda response must be valid json
 2024-05-09 16:38:56 127.0.0.1 - - [09/May/2024 16:38:56] "POST /bedrock/mistral HTTP/1.1" 502 -
[ERROR] Tests run: 4, Failures: 0, Errors: 4, Skipped: 0, Time elapsed: 128.5 s <<< FAILURE! -- in fr.simplex_software.quarkus_aws.tests.BedrockAiResourceIT
[ERROR] fr.simplex_software.quarkus_aws.tests.BedrockAiResourceIT.testWithMistral2 -- Time elapsed: 31.90 s <<< ERROR!
java.net.SocketTimeoutException: Read timed out

I've set the SAM_CLI_CONTAINER_CONNECTION_TIMEOUT environment variable to 60 seconds, as shown in the log file above, but it still raises the same exception. Running the same script (run-local.sh) locally works as expected. Here is this file:

#!/bin/bash
mvn -Durl=http://localhost:3000 clean install
sed -i 's/java11/java17/g' bedrock-gateway-api/target/sam.jvm.yaml
sam local start-api -t ./bedrock-gateway-api/target/sam.jvm.yaml --log-file ./bedrock-gateway-api/sam.log &
mvn -DskipTests=false failsafe:integration-test
docker run --name bedrock -p 8082:8082 --rm --network host nicolasduminil/bedrock-gateway-web:1.0-SNAPSHOT
./cleanup-local.sh

The exception is raised during the execution of the integration tests with the command mvn -DskipTests=false failsafe:integration-test.

What might be wrong here ?

Many thanks in advance.

Nicolas

Topics
Developer Tools
Tags
AWS CodeBuild
Language
English
asked 2 years ago293 views
1 Answer
0

I am unsure of the answer, but it looks like you are using Docker in CodeBuild and cannot connect Docker Damon.

Is PrivilegedMode enabled in CodeBuild? Maybe enabling it will improve the situation.

https://docs.aws.amazon.com/codebuild/latest/userguide/create-project-console.html#create-project-console-environment

EXPERT
answered 2 years ago
  • Nicolas Duminil
    2 years ago

    If you read my post, then you'll see that the mentioned exception is raised during the integration tests execution by Maven. So, it doesn't have anything to do with the docker run command as the execution stops before it. However, the sam local start-api command runs probably a Docker image. But since, as explained in my post, the mentioned script works as expected when run locally, it is supposed to run exactly the same way when run by CodeBuild. I think that you'll agree with me that a script, whatever it is, doesn't have to be modified in order to be run by CodeBuild. If it was the case, then CodeBuild wouldn't be usable. Anyway, to come back to the sam local start-api command and considering that it may execute a Docker image under the hood, there is no reason to try to modify the way that this image might by run and, above all, no reason to enable PriviledgedMode. Furthermore, there is no such an option enabling the Docker PriviledgedMode in the sam local start-api command. Perhaps, there is simply no running Docker daemon on the EC2 instance that CodeBuild spins up in order to run the build.I didn't find anything in the CodeBuild documentation talking about what processes are installed and running on these instances that CodeBuild is spinning up. But thank you anyway for your concern.

  • Akira EXPERT
    2 years ago

    Hello, nicolas! Please give me more time if you still need to solve the problem.

    Well, there is indeed an error before docker run. What I noticed is the following message.

    Timed out while attempting to establish a connection to the container.

    The message indicates that communication to the container has timed out.

    You are right that there is no clear reason to enable PrivilegedMode. However, it is worth a try.

    PrivilegedMode is not a SAM option, it is a CodeBuild setting, and in the CloudFormation template it is an Envelopment property.

    https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-codebuild-project-environment.html

    For an example of how to write it, please refer to the following. https://docs.aws.amazon.com/codebuild/latest/userguide/build-caching.html

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Relevant content