Kubernetes pod Operator with mwaa eks s3

0

Hi I am trying to run this in mwaa I am able to pull image form ecr repo and spin eks pod but I am getting error when its trying to run file inside that pod like cmds=["python","s3://mwaa-bucket/dags/test.py"]

I tried it with s3 URL and even giving s3 airflow dag folder path didn't work

Here is snippet for reference podRun = KubernetesPodOperator( namespace="mwaa", #image="ubuntu:18.04", image = "xxxxxxxxxx.dkr.ecr.us-east-2.amazonaws.com/new_repo:latest", cmds=["python","/usr/local/airflow/dags/test.py"], arguments=["-c", "ls"], labels={"foo": "bar"}, name="mwaa-pod-test", task_id="pod-task", get_logs=True, dag=dag, is_delete_operator_pod=False, config_file=kube_config_path, in_cluster=False, cluster_context='aws' )

2 個答案
1

The S3 bucket used by MWAA is not sync'd to the image used by Kubernetes. As such, your pod must get its code there itself.

See https://docs.aws.amazon.com/mwaa/latest/userguide/mwaa-eks-example.html

AWS
John_J
已回答 1 年前
0

In which section should I make a change for this in https://docs.aws.amazon.com/mwaa/latest/userguide/mwaa-eks-example.html let me know thanks for a response

已回答 1 年前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南