Understanding failed deployments

Question

I am new to Lightsail and I'm trying to debug a failed deployment, and I'm at the point of shooting in the dark. Any ideas would be appreciated!

I have two images: a Flask/Gunicorn image built from Python-alpine and an Nginx image. Locally, I can spin them up with `docker-compose` and they work beautifully.

But in Lightsail, all I know is that my Flask image "took too long":
```
[17/Mar/2023:24:11:33] [deployment:14] Creating your deployment
[17/Mar/2023:24:13:05] [deployment:14] Started 1 new node
[17/Mar/2023:24:14:39] [deployment:14] Started 1 new node
[17/Mar/2023:24:15:54] [deployment:14] Started 1 new node
[17/Mar/2023:24:16:14] [deployment:14] Took too long
```

Things I've tried that haven't worked:

From https://repost.aws/questions/QUrqo_fzNTQ5i1E08tT1uM7g/lightsail-container-took-too-long-to-deploy-all-of-a-sudden-nothing-in-logs:
- Set Gunicorn's logging to DEBUG. Sometimes I can see the Gunicorn process being killed by SIGTERM, but the "too long" part above has no additional information. 
- Set Health Check to 300 seconds in case that was the source of the SIGTERM. No effect.
- Increase capacity from "nano" to "micro" to "small". No effect.

From https://repost.aws/questions/QU8i3bF2BZQZiwKfxGw5CfgQ/how-to-deploy-amazon-linux-on-a-lightsail-container-service
- Made sure I pasted my launch command into the appropriate "launch command" form input. No effect.

Perhaps I missed something obvious.

**Update**:
I have Nginx configured to proxy requests to gunicorn and to serve static content. Below are Dockerfiles and docker-compose:

Flask/Gunicorn Dockerfile:
```
FROM python:3.10-alpine

ENV POETRY_VERSION=1.2.2 \
    POETRY_VIRTUALENVS_IN_PROJECT=true \
    PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1

RUN apk add --no-cache curl \
    && curl -sSL https://install.python-poetry.org | POETRY_VERSION=$POETRY_VERSION python3 -

WORKDIR /src

# TODO: build wheel for pipeline
COPY . .

RUN /root/.local/bin/poetry install --only main

CMD . /src/.venv/bin/activate && gunicorn -w 2 --log-level debug --bind=0.0.0.0:8080 'app:app'
```

Nginx Dockerfile:
```
FROM nginx:alpine

COPY ./nginx.conf /etc/nginx/nginx.conf
```

docker-compose.yml
```
version: "3.3"
services:
  web:
    image: myDockerHub/myImage
    restart: always
    volumes:
      - static_volume:/src/my_project/static
    ports:
      - "8080:80"

nginx:
    image: myDockerHub/nginx
    restart: always
    volumes:
      - static_volume:/src/my_project/static
    depends_on:
      - web
    ports:
      - "80:80"

volumes:
  static_volume:
```

Accepted Answer

Judging from absence of any meaningful log output from Gunicorn or Flask or nginx, I think there may be some entry point or launch command misconfiguration. It appears that, after 3 attempts to start a compute node running your containers, the system deemed this deployment a failure. Here's my attempt at standing up a configuration similar to what you're describing in your post. Here's the list of directories and files I've made, followed by the contents of each file: ``` static/index.html static/readme.txt hello.py Dockerfile-flask nginx.conf Dockerfile-nginx ``` ## `static/index.html:` ``` Static Content Root

Your static content is here.

``` ## `static/readme.txt:` ``` Did you read me? ``` ## `hello.py:` ``` from flask import Flask app = Flask(__name__) @app.route("/") def hello(): return "Hello, World!" ``` ## `Dockerfile-flask:` ``` FROM python:alpine RUN pip3 install Flask gunicorn RUN mkdir /app ADD hello.py /app # nginx container will be sending traffic here via localhost:5555, # see nginx.conf EXPOSE 5555 WORKDIR /app ENTRYPOINT ["gunicorn", "--bind", "localhost:5555", "--access-logfile", "-", "hello:app"] ``` ## `nginx.conf:` ``` server { listen 80; server_name localhost; location / { root /static; index index.html; } location /flask/ { # forward /flask/... requests to the gunicorn+flask container running on # the same compute node, listening on port 5555. proxy_pass http://localhost:5555/; } } ``` ## `Dockerfile-nginx:` ``` FROM nginx:alpine COPY static /static/ COPY nginx.conf /etc/nginx/conf.d/default.conf ``` Next, I've built the docker images locally. The one tagged `gunicorn-flask-app` is the flask application image. The one tagged `nginx-ingress` is the nginx image that has some static content and is configured to send some traffic to the gunicorn/flask container: ``` docker build . -t gunicorn-flask-app --platform=linux/amd64 -f Dockerfile-flask docker build . -t nginx-ingress --platform=linux/amd64 -f Dockerfile-nginx ``` Now let's create the container service: ``` aws lightsail create-container-service --service-name example --power nano --scale 1 ``` It takes some time for it to become ready for deployments. I use `aws lightsail get-container-services --service-name example` to see if it is ready, or check on it in Lightsail console. Let's push the locally built images to Lightsail. Push the flask image: ``` aws lightsail push-container-image --service-name example --label flask --image gunicorn-flask-app ``` Output (**note that the sequence number `48` in the image reference may be completely different for you**): ``` ... Digest: sha256:d46b4439329e8de4c608cf4f61865beea9b988fe732309c8a1ff0a83cd16b82c Image "gunicorn-flask-app" registered. Refer to this image as ":example.flask.48" in deployments. ``` Push the nginx image: ``` aws lightsail push-container-image --service-name example --label nginx --image nginx-ingress ``` Output: ``` ... Digest: sha256:bd008e587fcc59cc71f4421af0b0397a7a04d56e2ac2f4d0fc6e52a1adcc6ec6 Image "nginx-ingress" registered. Refer to this image as ":example.nginx.47" in deployments. ``` With all that done, we can now create a deployment: ``` aws lightsail create-container-service-deployment --service-name example \ --containers '{ "app": { "image": ":example.flask.latest" }, "ingress": { "image": ":example.nginx.latest", "ports": {"80": "HTTP"} } }' \ --public-endpoint '{"containerName": "ingress", "containerPort": 80}' ``` As deployment proceeds, I see logs like these for `app` container that runs gunicorn+flask application: ``` [deployment:3] Creating your deployment ... [1] [INFO] Starting gunicorn 20.1.0 ... [1] [INFO] Listening at: http://127.0.0.1:5555 (1) ... [1] [INFO] Using worker: sync ... [7] [INFO] Booting worker with pid: 7 [deployment:3] Reached a steady state ``` ... and these logs for `ingress` container that runs our nginx server: ``` [deployment:3] Creating your deployment /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/ ... /docker-entrypoint.sh: Configuration complete; ready for start up ... [notice] 1#1: using the "epoll" event method ... [notice] 1#1: nginx/1.23.3 ... [notice] 1#1: built by gcc 12.2.1 20220924 (Alpine 12.2.1_git20220924-r4) ... [notice] 1#1: OS: Linux 5.10.167-147.601.amzn2.x86_64 ... [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1024:4096 ... [notice] 1#1: start worker processes ... [notice] 1#1: start worker process 29 ... [notice] 1#1: start worker process 30 [deployment:3] Reached a steady state ``` When I visit my service via its public endpoint `https://example.qwerty1234567.us-west-2.cs.amazonlightsail.com/`, I can see that it returns the static content as expected. And when I visit `https://example.qwerty1234567.us-west-2.cs.amazonlightsail.com/flask/`, I see the hello world message from the Flask app. Note: `qwerty1234567.us-west-2` part of URL will differ for you, and will be something unique to your account and the region where you're deploying your service. You can find this URL in the container service details in Lightsail console or by running `aws lightsail get-container-services --service-name example`. # Prerequisites I am doing this on a Mac with Docker Desktop & Homebrew installed. In order to push locally built images for use in Lightsail container service deployments, you need to install: - AWS CLI: `brew install awscli` - lightsailctl: `brew install aws/tap/lightsailctl`

Understanding failed deployments

static/index.html:

static/readme.txt:

hello.py:

Dockerfile-flask:

nginx.conf:

Dockerfile-nginx:

Prerequisites

Relevant content

`static/index.html:`

`static/readme.txt:`

`hello.py:`

`Dockerfile-flask:`

`nginx.conf:`

`Dockerfile-nginx:`