aws mwaa does not see dags

Hi dmitrii,

It sounds like you've already checked the MWAA documentation and tried test DAGs—let’s build on that foundation to uncover the root cause. We’ll focus on verifying the S3 bucket setup, IAM permissions, and log configurations step by step.

Let’s dig into your issue now.

Clarifying the Issue: Diagnosing AWS MWAA Not Seeing DAGs

When AWS Managed Workflows for Apache Airflow (MWAA) doesn't show DAGs, even with all log groups empty, the issue often lies with one of the following areas: S3 bucket configuration, IAM roles, or DAG file validation.

1. Verify the S3 Bucket Setup

• Correct Bucket Path: Ensure that the DAGs are in the correct S3 prefix (dags/) as configured in MWAA.
• Permissions: Confirm that the MWAA Execution Role has:
  • s3:GetObject and s3:ListBucket permissions for the bucket and prefix.
  • Check for bucket policies blocking public access that might inadvertently restrict MWAA.
• File Format: DAG files must be .py files with no syntax errors.

Example Policy:

{
  "Effect": "Allow",
  "Action": ["s3:GetObject", "s3:ListBucket"],
  "Resource": [
    "arn:aws:s3:::<your-bucket-name>",
    "arn:aws:s3:::<your-bucket-name>/dags/*"
  ]
}

2. IAM Role Configuration

Verify that the Execution Role attached to MWAA has the necessary permissions. Attach the AmazonMWAAFullExecutionRole policy or customize it to include S3, CloudWatch, and MWAA service permissions.

Here’s a comprehensive IAM policy example for MWAA:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowS3AccessForDAGs",
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:ListBucket"],
      "Resource": [
        "arn:aws:s3:::<your-bucket-name>",
        "arn:aws:s3:::<your-bucket-name>/dags/*"
      ]
    },
    {
      "Sid": "AllowLoggingToCloudWatch",
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents",
        "logs:GetLogEvents"
      ],
      "Resource": "arn:aws:logs:<region>:<account-id>:log-group:airflow-*"
    },
    {
      "Sid": "AllowMWAAAccessToEnvironment",
      "Effect": "Allow",
      "Action": [
        "airflow:PublishMetrics",
        "airflow:GetEnvironment",
        "airflow:ListEnvironments"
      ],
      "Resource": "*"
    }
  ]
}

Replace:

• <your-bucket-name> with your S3 bucket name.
• <region> and <account-id> with your AWS region and account ID.

3. DAG File Validity

• Use airflow dags list locally or run the DAGs in a local Airflow environment to check for syntax errors or invalid task definitions.
• Ensure all dependencies referenced in DAG files are available in the environment, including Python libraries.
• Validate that all DAGs have a proper structure, including:

  from airflow import DAG
  from airflow.operators.dummy import DummyOperator
  from datetime import datetime
  
  with DAG("example_dag", start_date=datetime(2024, 1, 1), schedule_interval="@daily") as dag:
      task1 = DummyOperator(task_id="start")

4. Logs Configuration

If the MWAA logs remain empty, follow these troubleshooting steps:

1. Enable MWAA Logging:

   • Go to the MWAA Console → select your environment → under Monitoring, confirm that the logging options are enabled for:
     • Task logs
     • Worker logs
     • Scheduler logs

2. Validate CloudWatch Log Group Configuration:

   • Go to the CloudWatch Console → Logs → Log groups.
   • Ensure the following log groups exist:
     • /aws/airflow/<environment-name>/task
     • /aws/airflow/<environment-name>/worker
     • /aws/airflow/<environment-name>/scheduler
   • If the log groups don’t exist, ensure the MWAA Execution Role has permissions to create them.

3. Check Log Stream Creation:

   • Open the corresponding log group and look for active log streams. If no log streams exist:
     • Confirm MWAA can publish logs (logs:CreateLogStream and logs:PutLogEvents permissions).
     • Restart the MWAA environment to force log creation.

4. Inspect Airflow Logs Locally:

   • Run your DAGs locally in a non-MWAA Airflow environment to detect errors that prevent them from loading (e.g., Python syntax errors, missing dependencies).

5. MWAA Environment Health

• Verify that the MWAA environment is in a healthy state. Restart the environment if necessary.
• Check metrics in Amazon CloudWatch to confirm that MWAA is processing DAGs.

Common Pitfall: S3 Block Public Access

Even though MWAA doesn’t require public access, the S3 Block Public Access setting can sometimes overly restrict permissions. Double-check this setting to ensure it aligns with your access policies.

Conclusion

By addressing S3 setup, IAM roles, and log configurations while validating the DAGs, you should be able to identify and resolve the issue. Let me know if you need further assistance or clarification!

Cheers, Aaron 🚀😊

Hi all, similar issue here. I have tried all debugging steps. If someone can reproduce this issue I posted here please let me know the solution. https://github.com/aws-ia/terraform-aws-mwaa/issues/81