EndPoint Connection Error while connecting to AWS SecretManager from MWAA

0

I was just trying to complete the connection configuration for MWAA which has been specified in the article https://docs.aws.amazon.com/mwaa/latest/userguide/connections-secrets-manager.html but i'm ending up with the Endpoint Connection Error. Here is the Error log from Airflow "botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "https://secretsmanager.secretsmanager.amazonaws.com/"

2 Answers
0

Please ensure that the Secrets Manager endpoint is accessible via PrivateLink, Internet Gateway (IGW), and/or other routing, and that security groups and endpoint policies allow access, as well as that the MWAA execution IAM role is authorized to access Secrets Manager.

AWS
John_J
answered a year ago
  • I've attached the required policies and opened all traffic in the security groups. The example dag to read the variables stored in AWS Secretsmanager works fine. but for reading connections it's giving the above error! Here is some more info on error log, [2023-05-05, 11:49:43 UTC] {{base.py:73}} INFO - Using connection ID 'aws_default' for task execution. [2023-05-05, 11:49:45 UTC] {{taskinstance.py:1768}} ERROR - Task failed with exception Traceback (most recent call last): File "/usr/local/airflow/.local/lib/python3.10/site-packages/urllib3/connection.py", line 174, in _new_conn conn = connection.create_connection( File "/usr/local/airflow/.local/lib/python3.10/site-packages/urllib3/util/connection.py", line 72, in create_connection for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): File "/usr/lib/python3.10/socket.py", line 955, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known

0

Logs looks like connection error , usually hooks use aws_default connection to connect secretes manager or any AWS service ( if you use operator hook to connect). I believe your environment is private with out NAT , in such case connecting to endpoint secretsmanager.secretsmanager.amazonaws.com use public IP and it will not receive response from secretsmanager.secretsmanager.amazonaws.com as a workaround you can use endpoint secretsmanager.{region}.amazonaws.com (secretsmanager.us-east-2.amazonaws.com ) to communicate.

procedure to regional endpoint this :

  1. Update connection in webUI-->Connection-->AWS_Default-->Extras {"role_arn":"arn:aws:iam::AccountID:Role/rolename","region_name":"region like us-east-2"}

or

  1. Create a new connection with name aws-newdefault with connection type " aws web service" and in extras, provide below . {"role_arn":"arn:aws:iam::AccountID:Role/rolename","region_name":"region like us-east-2"}

and in your task code make sure you use conn_id=aws-newdefault

If your environment is not private then you need to make sure if there are no restrictions in networks like policies , NACL,SG ,TGW etc or do partial testing by creating Ec2 instance using same VPC and SG . SSH to ec2 and try to do telnet, Dig, nc with and without region to make sure your network is good.

Telnet secretsmanager.amazonaws.com telnet secretsmanager.amazonaws.com 443 nc -v secretsmanager.amazonaws.com 443 ping secretsmanager.amazonaws.com

similarly do above for secretsmanager.us-east-2.amazonaws.com

if you still see issue , I would suggest to open a support case.

AWS
answered a year ago
  • It turns out to be an error in the sample code provided in the AWS documentation, setting the parameter "client = hook.get_client_type()" or hardcoding the region for example "client = hook.get_client_type(region_name='us-east-1')" solved the issue!

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions