Load S3 Bucket data into Aurora PostgreSQL tables


I need to load my data from S3 Bucket into the Aurora PostgreSQL tables. I read https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_PostgreSQL.S3Import.html#USER_PostgreSQL.S3Import.Reference this documentation and I will try. But, do we have any other way to be able to handle this case?

Especially I want to ask, Is there any LOAD command which exists on Aurora PostgreSQL?


2 Answers

One option could be to use DMS.

DMS does support S3 as a source - https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.S3.html

DMS supports Aurora Postgres as a target - https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Target.PostgreSQL.html

profile pictureAWS
answered 2 years ago
  • I want to perform integrity check on my files in S3 bucket (The record count and md5 are 2 checks we need to do when files arrive in S3 from source DB). When these checks are successful, we want to load these files into Postgres. Can you please guide how this can be done? Integrity checks can be done in a bash script. Can we use the same script to copy files to Postgres?


The procedure for using the table_import_from_s3() function is explained here: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_PostgreSQL.S3Import.html

You call the function from a SELECT statement. And so you could automate that by calling psql from a shell script, and passing the SELECT statement as a -c (command) or -f (SQL script file) argument to psql.

answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions