By using AWS re:Post, you agree to the Terms of Use

OS in Batch compute environment suddenly changed?



I have multiple AWS accounts in which I use AWS Batch to run jobs. All Batch compute environments were created with the default EC2 AMI: ECS_AL1 (Amazon Linux). Here's part of the output from aws batch describe-compute-environments for one of my accounts:

        "type": "MANAGED",  
        "state": "ENABLED",  
        "status": "VALID",  
        "statusReason": "ComputeEnvironment Healthy",  
        "computeResources": {  
            "type": "EC2",  
            "minvCpus": 0,  
            "maxvCpus": 64,  
            "desiredvCpus": 0,  
            "instanceTypes": \[  
            "subnets": \[  
            "securityGroupIds": \[  
            "instanceRole": "arn:aws:iam::...",  
            "tags": {},  
            "ec2Configuration": \[  
                    "imageType": "ECS_AL1"  

In the past I could use yum, as you do in Amazon Linux, inside Batch jobs to install dependencies for my workloads.
Yesterday I noticed that yum doesn't work anymore. I also noticed that the OS in my compute environments changed! Now it's Debian. Here's the output of cat /etc/os-release which I ran inside one of my Batch jobs:

PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION="10 (buster)"

Why did the OS inside my Batch jobs change and how can I make it Amazon Linux again?

For more context, I don't start Batch jobs directly. Metaflow ( in a SageMaker Notebook instance does that for me.

asked 2 years ago23 views
1 Answer

Turns out Metaflow was the culprit. Metaflow's config has the following key which specifies the container image: METAFLOW_BATCH_CONTAINER_IMAGE.

answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions