Connection Timeout in Visual ETL Flow from SageMaker Lakehouse to S3

Question

hi friends

I am encountering a "Connect timed out" error when running a Visual ETL Flow in AWS SageMaker Studio (Unified). The data source is SageMaker Lakehouse, and the destination is Amazon S3.

Could you please advise what configurations or permissions need to be set to ensure a successful connection and data transfer?

```
org.apache.hadoop.hive.ql.metadata.HiveException: software.amazon.awssdk.core.exception.SdkClientException: Unable to execute HTTP request: Connect to glue.ap-southeast-1.amazonaws.com:443 [glue.ap-southeast-1.amazonaws.com/13.215.125.213, glue.ap-southeast-1.amazonaws.com/13.251.154.77, glue.ap-southeast-1.amazonaws.com/13.215.2.224, glue.ap-southeast-1.amazonaws.com/18.138.197.228, glue.ap-southeast-1.amazonaws.com/13.250.213.118, glue.ap-southeast-1.amazonaws.com/13.250.227.180] failed: Connect timed out
```

best regards

re:Post Agent · Answer

This connection timeout error when running a Visual ETL Flow in SageMaker Unified Studio suggests networking or VPC configuration issues. Here are several potential solutions:

1. **VPC Configuration**: If you're using SageMaker Studio in VPC-only mode, ensure your security groups are properly configured with outbound rules that allow HTTPS traffic (port 443) to AWS APIs, particularly to AWS Glue endpoints. The error specifically mentions being unable to connect to glue.ap-southeast-1.amazonaws.com:443.

2. **VPC Endpoints**: Verify that your VPC has the necessary endpoints configured for SageMaker API, SageMaker runtime, Amazon S3, and AWS Glue. These endpoints allow your resources to communicate with AWS services without requiring internet access.

3. **Internet Access**: If your SageMaker Studio resources require internet access, ensure your domain is connected to private subnets with a NAT gateway configured.

4. **IP Address Availability**: Confirm that your VPC has enough available IP addresses for the instances being used.

5. **DNS Settings**: Make sure DNS hostnames and DNS support are enabled in your VPC.

6. **CORS Policy**: For S3 access specifically, ensure that a proper CORS policy has been added to the S3 bucket you're trying to access.

7. **IAM Permissions**: Verify that the project role or access role has the necessary permissions to access both the SageMaker Lakehouse data and the target S3 bucket.

8. **Retry Mechanism**: Consider implementing retry logic in your ETL flow to handle transient connection issues.

If the issue persists after checking these configurations, you might need to restart the application or consult with your AWS administrator to review the network settings in more detail.  
**Sources**  
[Resolve connectivity issues with SageMaker Studio in VPC-only mode | AWS re:Post](https://repost.aws/knowledge-center/sagemaker-studio-jupyterlab-connect)  
[Amazon S3 data in Amazon SageMaker Unified Studio - Amazon SageMaker Unified Studio](https://docs.aws.amazon.com/sagemaker-unified-studio/latest/userguide/data-s3.html)  
[Using both external data and fine-grained data in Amazon SageMaker Unified Studio visual ETL flows - Amazon SageMaker Unified Studio](https://docs.aws.amazon.com/sagemaker-unified-studio/latest/userguide/visual-etl-combining-different-data.html)  
[Network issue to S3: Timeouts in S3? | AWS re:Post](https://repost.aws/questions/QUs9Xx9yWmTJOHk9FpwcRRbw/network-issue-to-s3-timeouts-in-s3)

Connection Timeout in Visual ETL Flow from SageMaker Lakehouse to S3

Add your answer

Relevant content