Redshift copy from s3 fails to connect

Question

When I execute a `copy table from 's3://...'  credentials 'aws_iam_role=' csv` in my serverless Redshift db, the operation seems to take a long time and finally fails with a connection timeout trying to reach S3.

See the error below:
```
ERROR: S3CurlException: Connection timed out after 50001 milliseconds, CurlError 28, multiCurlError 0, CanRetry 1, UserError 0 Detail: ----------------------------------------------- error: S3CurlException: Connection timed out after 50001 milliseconds, CurlError 28, multiCurlError 0, CanRetry 1, UserError 0 code: 9002 context: Listing bucket=my-bucket prefix=data/2022-11-01T00:00:00.000Z/test.csv query: 2602701 location: s3_utility.cpp:688 process: padbmaster [pid=24336] ----------------------------------------------- [ErrorId: 1-636d5c5a-0102430e2a36b8567a19097c]
```
Where should I look for more details to help me debug?

Are there additional networking settings or security group rules I should review for my Redshift workgroup?

Answer

Hi - A looks like based on description above that your redshift cluster may be not be using the internet if thats how you have set up. This may be causing redshift not being able to connect to S3. Also not sure if you have enhanced networking on or not. In Amazon Redshift, network traffic created by COPY, UNLOAD, and Amazon Redshift Spectrum flow through a network interface. This network interface is internal to the Amazon Redshift cluster, and is located outside of your Amazon Virtual Private Cloud (Amazon VPC). By default, the network traffic is then routed through the public internet to reach its destination [1]. Also if your Redshift cluster is in a VPC, you must configure the VPC to allow connections from the server that you are running the S3 Loader.

Reference :

Reference :
1. https://aws.amazon.com/premiumsupport/knowledge-center/redshift-enhanced-vpc-routing/
2. https://docs.aws.amazon.com/redshift/latest/mgmt/enhanced-vpc-routing.html

Redshift copy from s3 fails to connect

関連するコンテンツ