Airflow Web Server crashes when deploying

0

We are using Amazon Managed Workflows for Apache Airflow (MWAA) and we noticed that suddenly after some Variable creation and minor DAG update we can't access the Apache Airflow UI and getting the error message “ERR_EMPTY_RESPONSE” in the UI web page and while looking at the webserver console log we see the error:

Log stream name: webserver_console_ip-xx-x-xx-xxx.ec2.internal_xxxxxxxxxx.xxxxxxx.log

In script: sqlalchemy.py

In function: process_result_value

In line 176 "return type_map[data['type']](**data['attrs'])"

Error message:

KeyError: 'type'

The Error chain starts from “File "/usr/local/lib64/python3.7/site-packages/sqlalchemy/util/compat.py", line 178, in raise_”

Looking at the logs, the webserver tries to boot itself up and crashes repeatedly in the same line of code.

Important to note that we see that the environment is still working as some DAGs (but not all) are still completing their tasks (we are receiving emails of confirmation).

Before the error occurred, we created a new variable and used it in a DAG, I don’t believe this should affect the Web Server… The site should start normally even in a case where a DAG is "broken". We have not made any changes to the environment other than that.

已提问 2 年前771 查看次数
1 回答
0

If you are using requirements.txt it may be that one or more are causing conflicts as MWAA does not currently install requirements on the web server.

It may also be that newer Airflow packages may be trying to upgrade the Scheduler, which can be prevented by using --constraint in your requirements.txt

AWS
John_J
已回答 2 年前

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则