private mwaa environment: UI won't start

Question

Hello all,

I have  created private mwaa environment.
Airflow version = 2.8.1
Environment  class = mw1.small
Workers count with default values.
Region is eu-north-1

Added the following settings:

celery.sync_parallelism = 1 
core.dag_file_processor_timeout = 150
core.dagbag_import_timeout = 90 
core.min_serialized_dag_update_interval = 300
scheduler.dag_dir_list_interval = 600 
scheduler.min_file_process_interval = 300
scheduler.parsing_processes = 2 
scheduler.processor_poll_interval = 60

Added  inbound rules for HTTPS where source is both IPV4 and IPV6.

Here is the policy attached to the mwaa environment:

{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Effect": "Allow",
			"Action": "airflow:PublishMetrics",
			"Resource": "arn:aws:airflow:::environment/"
		},
		{
			"Effect": "Deny",
			"Action": "s3:ListAllMyBuckets",
			"Resource": [
				"arn:aws:s3:::",
				"arn:aws:s3:::/*"
			]
		},
		{
			"Effect": "Allow",
			"Action": [
				"s3:GetObject*",
				"s3:GetBucket*",
				"s3:List*"
			],
			"Resource": [
				"arn:aws:s3:::",
				"arn:aws:s3:::/*"
			]
		},
		{
			"Effect": "Allow",
			"Action": [
				"logs:CreateLogStream",
				"logs:CreateLogGroup",
				"logs:PutLogEvents",
				"logs:GetLogEvents",
				"logs:GetLogRecord",
				"logs:GetLogGroupFields",
				"logs:GetQueryResults"
			],
			"Resource": [
				"arn:aws:logs:::log-group:airflow--*"
			]
		},
		{
			"Effect": "Allow",
			"Action": [
				"logs:DescribeLogGroups"
			],
			"Resource": [
				"*"
			]
		},
		{
			"Effect": "Allow",
			"Action": "cloudwatch:PutMetricData",
			"Resource": "*"
		},
		{
			"Effect": "Allow",
			"Action": [
				"sqs:ChangeMessageVisibility",
				"sqs:DeleteMessage",
				"sqs:GetQueueAttributes",
				"sqs:GetQueueUrl",
				"sqs:ReceiveMessage",
				"sqs:SendMessage"
			],
			"Resource": "arn:aws:sqs::*:airflow-celery-*"
		},
		{
			"Effect": "Allow",
			"Action": [
				"kms:Decrypt",
				"kms:DescribeKey",
				"kms:GenerateDataKey*",
				"kms:Encrypt"
			],
			"NotResource": "arn:aws:kms:*::key/*",
			"Condition": {
				"StringLike": {
					"kms:ViaService": [
						"sqs..amazonaws.com"
					]
				}
			}
		}
	]
}

The Airflow UI can't be reached.
How can I fix it?

In general, how am I supposed to work with private mwaa environment and VPC?

Thanks in advanced.

re:Post Agent · Answer

To resolve the issue with your private Amazon Managed Workflows for Apache Airflow (MWAA) environment and access the Airflow UI, you need to consider several factors:

1. Network Configuration:
   - Ensure that your VPC network configuration allows access to the web server and metadatabase.
   - Check that your DNS settings are correct and that the Airflow UI's domain name resolves properly.
   - Verify that your network ACLs and route tables are configured according to Amazon MWAA security guidelines for Amazon VPC.
   - Modify your security groups to allow traffic on ports 443 and 5432.

2. VPC Endpoints:
   - For private MWAA environments, you need to set up VPC endpoints to allow communication between your VPC and MWAA services.
   - Create VPC endpoints for services that MWAA needs to interact with, such as Amazon S3, CloudWatch, Amazon SQS, and Amazon ECR.

3. Execution Role Permissions:
   - Review and update your execution role policy to ensure it has the necessary permissions for MWAA to access required AWS services.
   - Make sure the role has permissions for CloudWatch and Amazon S3 where logs are written.

4. Access Mechanism:
   - For private MWAA environments, you need a mechanism to access the VPC endpoint for the Apache Airflow Web server.
   - Consider using a Bastion Host or AWS Client VPN to access your private VPC resources.

5. Troubleshooting:
   - Run the Amazon MWAA verify script from AWS support tools to check your environment's health.
   - Check Apache Airflow logs in CloudWatch for any error messages or issues.

To work with a private MWAA environment and VPC:

1. Set up VPC endpoints for all necessary AWS services.
2. Use a Bastion Host or VPN solution to access your private VPC resources.
3. Ensure your VPC has the correct network configuration, including subnets, route tables, and security groups.
4. Use private PyPi repositories or bundle libraries as custom plugins if you need to install additional Python packages.
5. Regularly monitor your environment using CloudWatch and review logs for any issues.

By following these steps and ensuring your network configuration is correct, you should be able to access and work with your private MWAA environment.  
**Sources**  
[Troubleshoot Apache Airflow UI access issues in MWAA | AWS re:Post](https://repost.aws/knowledge-center/mwaa-airflow-access-issues)  
[Troubleshooting: Creating and updating an Amazon MWAA environment - Amazon Managed Workflows for Apache Airflow](https://docs.aws.amazon.com/mwaa/latest/userguide/t-create-update-environment.html)  
[Explore Amazon MWAA network architecture - Amazon Managed Workflows for Apache Airflow](https://docs.aws.amazon.com/mwaa/latest/migrationguide/mwaa-architecture.html)

private mwaa environment: UI won't start

Add your answer

Relevant content