Rollback docker component with AWS Greengrass V2: follow up question

0

Hey there,

I posted this question some days ago on the forum. The problem was that the Rollback function did not work as expected when deploying ECR docker containers through AWS Greengrass on my Raspberry Pi. When I deployed a deliberately non-working docker container, the deployment would not roll back to the previous component configuration but returen a SUCCESSFUL deployment and an UNHEALTHY device state.

As you can see in the post, I was suggested to run the docker container in detached mode and using "Startup" and "Shutdown" instead of "Run". My last answer however suggests that the Rollback still does not work as I would hope. Now the device stays healthy but there is no docker container running on the RPI after the faulty docker component errors out.

Can anyone help me with this issue?

Thanks a lot!

  • Can you share what the Startup section of the recipe looks like? In that there should be logic to start the container then wait a period time to determine if it's healthy. If so, you're last command in Startup should be an exit 0 (success), otherwise, exit 1 (or other non-zero number for ERRORED).

2 Answers
1

Since Docker -d will exit with a successful code immediately, startup isn't achieving anything in terms of rollback. You'd need the command to exit with a failure in order to rollback.

The better way to do this is use startup without -d which will keep docker in the foreground. Inside your application in the container, use Greengrass IPC to report that your component is healthy by using UpdateState with the RUNNING state: https://docs.aws.amazon.com/greengrass/v2/developerguide/ipc-component-lifecycle.html#ipc-operation-updatestate

If you do not report running within the startup timeout your component will move to ERRORED, then retry 2 more times before becoming BROKEN and triggering the rollback.

Cheers,

Michael

AWS
EXPERT
answered a year ago
  • sorry for coming back to you so late. so your colleague stated that I should run the docker component "in background mode", which I assumed to be the detached (-d) mode. but apparently, I was wrong?

    I am running a python script inside my docker container. I haven't looked too much into IPC but I will have to install greengrass on my docker image additionally to use it I guess. All I really want to do is to start the docker container and run a python file therein. If the python script fails, usually the docker container shuts down. So do I really need to make use of IPC?

    If I do need to make use of IPC for the device to know that the component is healthy and running... how do I do this within the python script? could you maybe show me a small example?

0

Even when using -d, docker run would exit with 127 status if the command to be execute cannot be found. In your case, the container might find the command to execute but not the some other dependent file. In this case the container starts and docker exits with status 0, but then the container fails because of some other error. Greengrass IPC in your code is the recommended solution in case you are ok in creating a dependency in your container to run in a Greengrass environment. In case you would like to have a container which is agnostic of running in Greengrass or not, you could also add a small watchdog/monitor code that checks if the container is still running after X seconds and exists with non-zero status otherwise.

AWS
EXPERT
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions