Importing data from Amazon S3 .csv file

Question

Hello All!

Would like to Importing data from Amazon S3 into an RDS for PostgreSQL DB instance, but something is wrong from my side:

ERROR:  HTTP -1. curlCode: 28, Timeout was reached
CONTEXT:  SQL function "table_import_from_s3" statement 1

SQL state: XX000

I used this links to help me, but same error.

https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_PostgreSQL.S3Import.html
https://stackoverflow.com/questions/58616005/import-postgres-data-into-rds-using-s3-and-aws-s3
https://repost.aws/knowledge-center/aurora-postgresql-integrate-s3

RDS Postgresql 16.1, i've test in to import the csv file from local machine and works well, but when i trie to do the s3, no.

Accepted Answer

Using the IMPORT FROM S3 function, RDS PostgreSQL can directly read a file from an S3 bucket and import the data. This approach works well if your RDS PostgreSQL instance has a network path to access the S3 bucket. However, if your instance is private and the subnet route table lacks a NAT gateway or NAT server, then direct access to the S3 bucket won't be possible. An alternative solution is to leverage AWS PrivateLink S3 Gateway, which allows data to flow internally through a VPC endpoint.

Answer

Hello.

Since it was a timeout error, I thought there was a problem with the network settings for accessing S3 from RDS.  
So, I thought that if I created a gateway VPC endpoint for S3 and set a route to the VPC endpoint in the route table of the subnet used by RDS, I would be able to communicate.  
https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints-s3.html#create-gateway-endpoint-s3

Importing data from Amazon S3 .csv file

Contenuto pertinente