Deploying a Dockerised go server to AWS Elasticbeanstalk/Ec2 instance via GitHub Actions environment variable issue

0

My deployment stack looks like this:

Go project that has a Dockerfile and config.yml for GitHub automation. Simple AWS setup where I build to ECR and deploy to my EC2 instance via Elastic Beanstalk All my AWS credentials are stored in Github Secrets. Previously the rest of the required env variables were stored in config.env in the Go project. All the above automation works just fine.

For better security, easier modification with branches, etc I want to move all the information out of the config.env file into the Secrets and Variables of the GitHub Action.

Unfortunately, these are not parsing through to the env variables of the Docker container as expected. I tried for a few hours last night and have got to a point where I am stumped and its just not making sense.

So this is not the complete set of variables, but it is enough to show where I am stumped.

First Step - .elasticbeanstalk\deploy.yml for GitHub For testing purposes, I have set one env manually and one from GitHub variables (I get the same result with secrets.name too).

${{ secrets.AWS_REGION }} etc work as expected

Both the echo's in the run block show up in the GitHub Action logs just fine as expected. So it tells me I am pulling the Variables/Secrets just fine at this point. Both --build-arg seem to work as expected too as you will see in the next part.

name: Build and Push to ECR

on:
  push:
    branches:
      - main 

jobs:
  build:
    runs-on: ubuntu-latest
    env:
      DB_USER: ${{ vars.DB_USER }}
      DB_NAME: tickerdb

    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Login to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v1
        env:
          AWS_REGION: ${{ secrets.AWS_REGION }}
          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

      - name: Build, tag, and push image to Amazon ECR
        env:
          ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          ECR_REPOSITORY: tickertime-backend-app
          IMAGE_TAG: latest
        run: |
          echo "DB_USER from actions: $DB_USER"
          echo "DB_NAME from actions: $DB_NAME"
          docker build --build-arg DB_USER=$DB_USER --build-arg DB_NAME=$DB_NAME -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
          docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG

Second Step - Dockerfile for building the container Once again I set one of the ARG and one of the ENV manually to test. All three echos also show up correctly in the GitHub Action logs telling me that the ARG of DB_USER is being passed in correctly at this point.

FROM golang:1.19.1

WORKDIR /app

COPY go.mod go.sum ./

# Set arguments for the build
ARG DB_USER
ARG DB_NAME
ARG DB_PORT=5454

# Set the environment variable
ENV DB_HOST=mydb.hostname.ap-southeast-2.rds.amazonaws.com
ENV DB_PORT=$DB_PORT
ENV DB_NAME=$DB_NAME
ENV DB_USER=$DB_USER

RUN echo "DB_HOST from dockerfile: $DB_HOST"
RUN echo "DB_PORT from dockerfile: $DB_PORT"
RUN echo "DB_USER from dockerfile: $DB_USER"
RUN echo "DB_NAME from dockerfile: $DB_NAME"

RUN go mod download

COPY . .

RUN go build -o main .

EXPOSE 8080

CMD ["./main"]

Last Step - Inside the Docker container BUT when I check the env variables of my Docker container the manual persists, but not the one that has been passed in from the start (DB_HOST).

func main() {
    log.Println("DB_HOST:", os.Getenv("DB_HOST"))
    log.Println("DB_PORT:", os.Getenv("DB_PORT"))
    log.Println("DB_NAME:", os.Getenv("DB_NAME"))
    log.Println("DB_USER:", os.Getenv("DB_USER"))

All of them come through fine except DB_USER. At this point, I am stumped and have yet to find anything online that helps.

Any help would be much appreciated :)

  • Curious to know whether renaming DB_USER to DB_USER_1 allows your Go program to pick it up and print it. If it does, it would suggest that there's something else in your environment that is causing DB_USER to be overridden. Also, could you explicitly try: CMD DB_USER=... && ./main instead of what you have?

asked 21 days ago128 views
1 Answer
0

Hi

you're defining environment variables for the build job (DB_USER and DB_NAME) and then using them during the build process. However, inside the Dockerfile, you're also defining an environment variable for DB_HOST.

Option 1:

In your Github Actions workflow, add DB_HOST to the environment variables with the desired value:

env:
  DB_USER: ${{ vars.DB_USER }}
  DB_NAME: tickerdb
  DB_HOST: your_db_host  # Replace with your actual DB host

DockerFile:

ENV DB_PORT=$DB_PORT
ENV DB_NAME=$DB_NAME
ENV DB_USER=$DB_USER

Pass DB_HOST as a build argument during the build step:

      - name: Build, tag, and push image to Amazon ECR
        ...
        run: |
          docker build --build-arg DB_USER=$DB_USER --build-arg DB_NAME=$DB_NAME --build-arg DB_HOST=$DB_HOST -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .

Option 2: Use a multi-stage build

FROM golang:1.19.1 AS builder

WORKDIR /app

COPY go.mod go.sum ./

# Set arguments for the build
ARG DB_USER
ARG DB_NAME
ARG DB_PORT=5454

# Set the environment variable
ENV DB_HOST=mydb.hostname.ap-southeast-2.rds.amazonaws.com
ENV DB_PORT=$DB_PORT
ENV DB_NAME=$DB_NAME
ENV DB_USER=$DB_USER

RUN go mod download

COPY . .

RUN go build -o main .

# Final stage - copy the binary and environment variables
FROM alpine:latest

COPY --from=builder /app/main /app/main
ENV DB_HOST=$DB_HOST
ENV DB_PORT=$DB_PORT
ENV DB_NAME=$DB_NAME
ENV DB_USER=$DB_USER

EXPOSE 8080

CMD ["./main"]

This approach builds the application in a separate stage with the environment variables defined. Then, it copies only the binary and the environment variables to the final, lightweight image. try it and lets see

profile picture
EXPERT
GK
answered 21 days ago
profile picture
EXPERT
reviewed 21 days ago
  • Thanks for that super fast response GK.

    Sorry typo there. Its DB_USER that isn't coming through (have updated my original question).

    env: DB_USER: ${{ vars.DB_USER }}

    Defined right from the start and appears in all the echos but not in the last part when I check the env variable within Go

  • Hi Mike, Let me know if you are successful otherwise, please comment the issue again Thanks

  • Still no luck on this GK.

    Option 1 As mentioned above DB_USER is the one not coming through correctly so this doesn't help.

    Option 2 This returns an error during the build ERROR: failed to solve: builder: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

    One other piece of information to note. This only happens when I use GitHub to build. If I am on my local machine and use: docker build --build-arg DB_USER=tickertime -t backend-app . It passes the environment variable across just fine into the container with my original Dockerfile

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions