Parallel load support in aws_s3 extension to import data in Aurora PostgreSQL

0

A customer wants to know whether there is any native mechanism to parallel upload data to Aurora from multiple csv files in S3. Based on the documentation [1] unlike, DMS there is no supported way of doing it. Secondly, is there a way one can modify aws_s3 extension i.e. wondering if have open-sourced it. DMS is another alternative, but wanted to rule out if there I am missing anything specific here.

[1] https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraPostgreSQL.Migrating.html#USER_PostgreSQL.S3Import

1 Answer
0
Accepted Answer

As the document you mentioned, aws_s3 uses S3 API to download a file and then uses COPY statement to load data. It supports a single file, not multiple files as input. The customer need to implement to handle multiple files in their application.

aws_s3 is released by RDS/Aurora PostgreSQL team and not seemed to be open-sourced.

answered 4 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions