Airflow 2.0 database initialisation error

Hello,

we were testing the new Airflow 2.0 implementation but the new v2.0.2 version fails during the database initialisation (webserver logs see below). Do we need to update some other settings than the Airflow version when upgrading from Airflow v1 to v2? We already tried destroying and re-applying. On our local docker-compose setup (official v2 docker-compose), everything works. Thank you for your support!

Our configuration:

resource "aws_mwaa_environment" "airflow" {
execution_role_arn = aws_iam_role.airflow.arn
name = "<fancyname>"
airflow_version = "2.0.2"

source_bucket_arn = aws_s3_bucket.airflow.arn
dag_s3_path = aws_s3_bucket_object.dags.key
requirements_s3_path = "requirements.txt"
plugins_s3_path = "plugins.zip"

network_configuration {
security_group_ids = [aws_security_group.airflow.id]
subnet_ids = var.private_subnets
}

airflow_configuration_options = {
"secrets.backend" = "airflow.contrib.secrets.aws_secrets_manager.SecretsManagerBackend"
}

webserver_access_mode = "PUBLIC_ONLY"
}

And the logs:

2021-05-28T22:43:41.894+02:00 WARNING:root:Failed to log action with (psycopg2.errors.UndefinedTable) relation "log" does not exist
2021-05-28T22:43:42.104+02:00 LINE 1: INSERT INTO log (dttm, dag_id, task_id, event, execution_dat...
2021-05-28T22:43:42.206+02:00 ^
2021-05-28T22:43:42.340+02:00 2021-05-28T22:43:42.409+02:00 [SQL: INSERT INTO log (dttm, dag_id, task_id, event, execution_date, owner, extra) VALUES (%(dttm)s, %(dag_id)s, %(task_id)s, %(event)s, %(execution_date)s, %(owner)s, %(extra)s) RETURNING log.id]
2021-05-28T22:43:42.595+02:00 [parameters: {'dttm': datetime.datetime(2021, 5, 28, 20, 43, 41, 872768, tzinfo=Timezone('UTC')), 'dag_id': None, 'task_id': None, 'event': 'cli_webserver', 'execution_date': None, 'owner': 'airflow', 'extra': '{"host_name": "ip-10-0-28-2.eu-central-1.compute.internal", "full_command": "['/usr/local/bin/airflow', 'webserver']"}'}]
2021-05-28T22:43:42.739+02:00 (Background on this error at: http://sqlalche.me/e/13/f405)
2021-05-28T22:43:42.835+02:00 /usr/local/lib/python3.7/site-packages/airflow/settings.py:385 DeprecationWarning: session_lifetime_days option from \[webserver] section has been renamed to session_lifetime_minutes. The new option allows to configure session lifetime in minutes. The force_log_out_after option has been removed from \[webserver] section. Please update your configuration.
2021-05-28T22:43:42.902+02:00 ____________ _____________
2021-05-28T22:43:43.029+02:00 ____ |( )_______ / /________ __
2021-05-28T22:43:43.110+02:00 ____ /| |_ /__ / / __ / __ _ | /| / /
2021-05-28T22:43:43.234+02:00 ___ ___ | / _ / _ / _ / / // / |/ |/ /
2021-05-28T22:43:43.371+02:00 // |// // // // _/____/|__/
2021-05-28T22:43:43.770+02:00 [2021-05-28 20:43:42,064] {{dagbag.py:451}} INFO - Filling up the DagBag from /dev/null
2021-05-28T22:43:43.885+02:00 [2021-05-28 20:43:42,067] {{logging_config.py:46}} INFO - Successfully imported user-defined logging config from log_config.LOGGING_CONFIG
2021-05-28T22:43:43.992+02:00 WARNING:flask_appbuilder.security.manager:No user yet created, use flask fab command to do it.
2021-05-28T22:43:51.448+02:00 Traceback (most recent call last):
2021-05-28T22:43:51.935+02:00 File "/usr/local/lib64/python3.7/site-packages/sqlalchemy/engine/base.py", line 1277, in execute_context
2021-05-28T22:43:52.235+02:00 cursor, statement, parameters, context
2021-05-28T22:43:52.535+02:00 File "/usr/local/lib64/python3.7/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
2021-05-28T22:43:52.734+02:00 cursor.execute(statement, parameters)
2021-05-28T22:43:52.798+02:00 psycopg2.errors.UndefinedTable: relation "dag" does not exist
2021-05-28T22:43:52.907+02:00 LINE 2: FROM dag
2021-05-28T22:43:53.269+02:00 ^
2021-05-28T22:43:53.328+02:00 2021-05-28T22:43:53.381+02:00 2021-05-28T22:43:53.445+02:00 The above exception was the direct cause of the following exception:
2021-05-28T22:43:53.501+02:00 2021-05-28T22:43:53.563+02:00 Traceback (most recent call last):
2021-05-28T22:43:53.636+02:00 File "/usr/local/bin/airflow", line 8, in <module>
2021-05-28T22:43:53.716+02:00 sys.exit(main())
2021-05-28T22:43:53.779+02:00 File "/usr/local/lib/python3.7/site-packages/airflow/main.py", line 40, in main
2021-05-28T22:43:53.843+02:00 args.func(args)
2021-05-28T22:43:54.124+02:00 File "/usr/local/lib/python3.7/site-packages/airflow/cli/cli_parser.py", line 48, in command
2021-05-28T22:43:54.182+02:00 return func(*args, **kwargs)
2021-05-28T22:43:54.229+02:00 File "/usr/local/lib/python3.7/site-packages/airflow/utils/cli.py", line 89, in wrapper
2021-05-28T22:43:54.301+02:00 return f(*args, **kwargs)
2021-05-28T22:43:54.357+02:00 File "/usr/local/lib/python3.7/site-packages/airflow/cli/commands/webserver_command.py", line 360, in webserver
2021-05-28T22:43:54.423+02:00 app = cached_app(None)
2021-05-28T22:43:54.468+02:00 File "/usr/local/lib/python3.7/site-packages/airflow/www/app.py", line 145, in cached_app
2021-05-28T22:43:54.530+02:00 app = create_app(config=config, testing=testing)
2021-05-28T22:43:54.590+02:00 File "/usr/local/lib/python3.7/site-packages/airflow/www/app.py", line 133, in create_app
2021-05-28T22:43:54.645+02:00 sync_appbuilder_roles(flask_app)
2021-05-28T22:43:54.699+02:00 File "/usr/local/lib/python3.7/site-packages/airflow/www/app.py", line 65, in sync_appbuilder_roles
2021-05-28T22:43:54.765+02:00 security_manager.sync_roles()
2021-05-28T22:43:54.812+02:00 File "/usr/local/lib/python3.7/site-packages/airflow/www/security.py", line 574, in sync_roles
2021-05-28T22:43:54.879+02:00 self.create_dag_specific_permissions()
2021-05-28T22:43:54.966+02:00 File "/usr/local/lib/python3.7/site-packages/airflow/www/security.py", line 526, in create_dag_specific_permissions
2021-05-28T22:43:55.023+02:00 .filter(or(models.DagModel.is_active, models.DagModel.is_paused))
2021-05-28T22:43:55.135+02:00 File "/usr/local/lib64/python3.7/site-packages/sqlalchemy/orm/query.py", line 3373, in all
2021-05-28T22:43:55.404+02:00 return list(self)
2021-05-28T22:43:55.460+02:00 File "/usr/local/lib64/python3.7/site-packages/sqlalchemy/orm/query.py", line 3535, in iter
2021-05-28T22:43:55.541+02:00 return self._execute_and_instances(context)
2021-05-28T22:43:55.606+02:00 File "/usr/local/lib64/python3.7/site-packages/sqlalchemy/orm/query.py", line 3560, in _execute_and_instances
2021-05-28T22:43:55.663+02:00 result = conn.execute(querycontext.statement, self._params)
2021-05-28T22:43:56.560+02:00 File "/usr/local/lib64/python3.7/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
2021-05-28T22:43:59.666+02:00 return meth(self, multiparams, params)
2021-05-28T22:43:59.729+02:00 File "/usr/local/lib64/python3.7/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
2021-05-28T22:44:00.801+02:00 return connection._execute_clauseelement(self, multiparams, params)
2021-05-28T22:44:02.608+02:00 File "/usr/local/lib64/python3.7/site-packages/sqlalchemy/engine/base.py", line 1130, in _execute_clauseelement
2021-05-28T22:44:02.663+02:00 distilled_params,
2021-05-28T22:44:02.806+02:00 File "/usr/local/lib64/python3.7/site-packages/sqlalchemy/engine/base.py", line 1317, in execute_context
2021-05-28T22:44:02.862+02:00 e, statement, parameters, cursor, context
2021-05-28T22:44:05.052+02:00 File "/usr/local/lib64/python3.7/site-packages/sqlalchemy/engine/base.py", line 1511, in handle_dbapi_exception
2021-05-28T22:44:05.176+02:00 sqlalchemy_exception, with_traceback=exc_info[2], from=e
2021-05-28T22:44:05.241+02:00 File "/usr/local/lib64/python3.7/site-packages/sqlalchemy/util/compat.py", line 182, in raise
2021-05-28T22:44:05.289+02:00 raise exception
2021-05-28T22:44:05.351+02:00 File "/usr/local/lib64/python3.7/site-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
2021-05-28T22:44:05.409+02:00 cursor, statement, parameters, context
2021-05-28T22:44:06.122+02:00 File "/usr/local/lib64/python3.7/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
2021-05-28T22:44:06.170+02:00 cursor.execute(statement, parameters)
2021-05-28T22:44:06.274+02:00 sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedTable) relation "dag" does not exist
2021-05-28T22:44:06.831+02:00 LINE 2: FROM dag
2021-05-28T22:44:06.884+02:00 ^
2021-05-28T22:44:06.940+02:00 2021-05-28T22:44:07.210+02:00 [SQL: SELECT dag.dag_id AS dag_dag_id
2021-05-28T22:44:07.272+02:00 FROM dag
2021-05-28T22:44:07.817+02:00 WHERE dag.is_active OR dag.is_paused]
2021-05-28T22:44:07.872+02:00 (Background on this error at: http://sqlalche.me/e/13/f405)