- 最新
- 最多得票
- 最多評論
MWAA has a maximum of 4 vCPUs (for a large environment). As such, you essentially have 4 parallel processes. Exceeding the Airflow default scheduler.parsing_processes of 2 will leave no resources left to actually schedule any tasks or update the heartbeat, as you're encountering with your error.
Creating session with aws_access_key_id
is an INFO level log item in the Airflow Amazon provider package and not specific to MWAA nor attached to any error condition.
Airflow is highly sensative to DAG parsing, as it runs Python import on every Python file. Recommend using .airflowignore to avoid parsing files that do not create top-level DAG objects and remove top-level code wherever possible (i.e. most code should be executed in a task, rather than by the parsing action--see example here)
相關內容
- AWS 官方已更新 8 個月前
- AWS 官方已更新 1 年前
- AWS 官方已更新 2 年前
- AWS 官方已更新 2 年前