Observations and questions around MWAA in Cloudformation

Question

Hi, before I dive into my question, I first wanted to share some observations I made working with the cloudformation template, which might be useful for other people, too.  The template can be found here:    
  
**AirflowConfigurationOptions**: as mentioned in other threads, don't try to configure the secrets manager backend until a fix from the AWS team is in place, right now it does break the environment.  
  
**DagS3Path/PluginsS3Path/RequirementsS3Path**: the later ones state that they need an s3 URI (s3:/bucket/path/), that is not the case. They are just paths relative to the bucket defined in _SourceBucketArn_.  
  
**EnvironmentClass**: the cloudformation docs don't specify the possible values for this and some of the screenshots in the mwaa docs show wrong keys in the UI. The right possibilities as of now are _mw1.small/mw1.medium/mw1.large_.  
  
**LoggingConfiguration > ModuleLoggingConfiguration**: The docs say you can configure CloudWatchLogGroupArn, that did not work for me it was silently ignored. In fact at first my role did not have create log group permissions and I did get errors in cloud trail. It will always try to create log groups following this pattern arn:aws:logs:::log-group:airflow--\[DagProcessing|WebServer|Task|Worker|Scheduler] - so make sure it has permissions to create and interact with these.  
  
**WebserverAccessMode**: Possible values are not in the cloudformation docs, but can be found in the cli docs: _PRIVATE_ONLY_ and _PUBLIC_ONLY_.  
  
**SourceBucketArn**: Bucket name needs to start with airflow. This one was difficult to find, ended up checking the raw cloudformation schema for the pattern. "^arn:aws(-_)?:s3:::airflow-_$" (sorry for the syntax but square brackets are considered links)  
  
Which brings me finally to my question, which relates to the next step, i.e., using WebserverAccessMode PRIVATE_ONLY and adding (in cloudformation or in my case cdk_cloudformation) an application load balancer, without needing to hardcode URLs after creation. The cloudformation docs state there is a _parameter_ WebserverUrl, which confuses me. How can this be a _parameter_? It feels this should be a _return value+ and show what is visible in the UI either as public endpoint or private VPC endpoint. Has anybody worked with this? Following the docs to get the URL I would most likely end up using a custom clouformation resource and calling get-environment to get the URL, because I cannot see how I could use WebserverURL as a parameter in a meaningful way.  
  
Also for context, I have not found a way to restrict the public endpoint created with PUBLIC_ONLY IP wise. It is protected with IAM authentication, but it feels wrong to have an internal tool not restricted by IP whitelist.  
  
Edited by: andreaslang on Jan 14, 2021 1:57 AM - just added info about s3 bucket

Answer

Hi!  
  
Thank you for your detailed analysis! The MWAA team will work to make these changes/additions to the documentation.   
  
You are correct that the Webserver URL is a return value, not something that can be set.  
  
While you can't limit the incoming address with the public web server option, you can restrict the ability to create the web token to a specific IP address or range, which provides the same effect, by adding an IP address condition ( https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_condition_operators.html#Conditions_IPAddress ) to the web UI access policy ( https://docs.aws.amazon.com/mwaa/latest/userguide/manage-access.html#web-ui-access )  
  
Thanks!  
  
Edited by: JohnJ-AWS on Jan 14, 2021 9:21 AM

Answer

Ah so the WebserverUrl is already a cloudformation return value? I will give this a shot then and see if I can use it. The web token creation IP limit condition can be the backup plan.  
  
Thank you

Observations and questions around MWAA in Cloudformation

Relevant content