Hello,

Please follow below steps

1. Once the docker image is pushed to ECR, you will need to make a Docker Registry connection on Airflow UI. Make sure to choose connection type as "Docker Registry" and Username as "AWS". Password is the string that you get when running "aws ecr get-login-password --region us-east-1"

2. Write your DAG, and specify the docker_conn_id with the name you chose above. Make sure your provided docker URL is correct. Also, make sure that the docker host has access from the MWAA Security group.

Below is the sample Dag

from airflow import DAG from datetime import datetime, timedelta import json, csv, requests from airflow.operators.docker_operator import DockerOperator

default_args = { "owner": "xxxxx", "start_date": datetime(2020, 11, 29), "depends_on_past": False, "email_on_failure": False, "email_on_retry": False, "email": "xxxx@xxx.com", "retries": 1, "retry_delay": timedelta(minutes=5) }

with DAG(dag_id="docker-test", schedule_interval="@hourly", default_args=default_args, catchup=False) as dag:

docker_test_task = DockerOperator(
    task_id='docker_task',
    image='123456789012.dkr.ecr.us-east-1.amazonaws.com/airflow-docker-rep:latest',
    execution_timeout=timedelta(minutes=30),
    docker_conn_id='docker',
    api_version='auto',
    docker_url='tcp://ec2-11-222-333-444.compute-1.amazonaws.com:1111'
)

docker_test_task

thanks for answer

Hello, Please clarify the "docker host has access from the MWAA Security group". Also, could you give us information about where to get the docker_url : docker_url='tcp://ec2-11-222-333-444.compute-1.amazonaws.com:1111'