Skip to content

Disable all Custom CloudWatch Metrics for MWAA

0

I want to disable all the custom cloudwatch metrics generate by my airflow environment.

I read about the metrics.metrics_block_list config here. So I added

metrics_block_list = scheduler,executor,dagrun,dag,pool,triggerer,celery,ti,task

(not sure if those are all relevant prefixes) as a custom config to the environment. But i still get the metrics.

Then i tried

metrics.metrics_use_pattern_match = true
metrics.metrics_block_list = ^.*$

But this does not have an effect either. Also, wenn locking in the config in the UI (by setting webserver.expose_config to true) i see that the value are still empty / false.

What am I missing?

3 Answers
5

You may try this:

  1. Use Environment Variables Instead of airflow.cfg MWAA prefers environment variables for config overrides. Try setting:
AIRFLOW__METRICS__METRICS_BLOCK_LIST=^.*$
AIRFLOW__METRICS__METRICS_USE_PATTERN_MATCH=true

Add these as custom environment variables in the MWAA console. 2. Double-Check MWAA Version Compatibility Some metrics settings are only supported in Airflow 2.3+. Confirm your MWAA environment is running a compatible version. 3. Restart the Environment After applying config changes, restart the MWAA environment to ensure they take effect. 4. Verify with CloudWatch Logs Check the MWAA logs in CloudWatch to confirm whether metrics are still being emitted. Sometimes metrics are cached or delayed.

EXPERT

answered a year ago

0
Accepted Answer

Setting metrics.statsd_on = False disabled all the custom metrics.

answered a year ago

0

Hi, thank you for the answer

  1. where can i set environment variables explicitly? afaik the environment config (e.g. in the porlta) is converted to environment variables. Thats what i am doing.
  2. Version 2.10, so this should be fine
  3. Restart was done
  4. Metrics are still there

So in general you say the config itself should be fine?

answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.