MWAA metrics export for data analytics

There are methods to query the metadata in the back-end database used by Amazon MWAA.

I recommend reviewing this blog, but not utilizing the AWS CloudFormation template as it will create an Amazon MWAA Enviroment.

https://aws.amazon.com/blogs/big-data/persist-and-analyze-metadata-in-a-transient-amazon-mwaa-environment/

By utilizing AWS Glue along with the example DAGs, your metadata will existing within Amazon S3 and can then be downloaded or queried by Amazon Athena. If you decide to utilize Amazon Athena, then you can then connect from Tableau by following this blog post example: https://aws.amazon.com/blogs/big-data/building-aws-data-lake-visualizations-with-amazon-athena-and-tableau/

Yes, you can export MWAA metrics for data analytics! Here's how:

The metrics you're looking for (DAG start/end times, task timings, status, etc.) are stored in MWAA's underlying metadatabase.

To export these metrics, you'll need to create a DAG that queries the metadatabase, as the database is only accessible from within the MWAA environment.

AWS provides official code examples for this purpose:

For exporting DAG run information to CSV: https://docs.aws.amazon.com/mwaa/latest/userguide/samples-dag-run-info-to-csv.html

Important Note: Since the MWAA metadatabase is not accessible outside its environment, the only way to extract this data is through a DAG running within MWAA. Once exported, you can then load this data into Tableau for your reporting needs.