Parallel load support in aws_s3 extension to import data in Aurora PostgreSQL
A customer wants to know whether there is any native mechanism to parallel upload data to Aurora from multiple csv files in S3. Based on the documentation 1 unlike, DMS there is no supported way of doing it. Secondly, is there a way one can modify aws_s3 extension i.e. wondering if have open-sourced it. DMS is another alternative, but wanted to rule out if there I am missing anything specific here.
As the document you mentioned, aws_s3 uses S3 API to download a file and then uses COPY statement to load data. It supports a single file, not multiple files as input. The customer need to implement to handle multiple files in their application.
aws_s3 is released by RDS/Aurora PostgreSQL team and not seemed to be open-sourced.
Changes to connection handling with Aurora Postgres?Accepted Answerasked 3 years ago
How to import postgresql or nosql datasets in Amazon Sagemaker?asked 5 months ago
Parallel load support in aws_s3 extension to import data in Aurora PostgreSQLAccepted AnswerEXPERTasked 3 years ago
PostGIS and Aurora compatibility: experience feedbackAccepted AnswerEXPERTasked 3 years ago
Aurora PostgreSQL aws_s3.query_export_to_s3 compressionasked a month ago
How to ensure replica is in sync between AWS RDS PSQL primary DB and Aurora RDS PSQL replica cluster?asked a day ago
Quicksight export more than 1 million rows data to multiple csvasked 3 months ago
Migration from Aurora PostgreSQL back to regular RDSAccepted Answerasked 3 years ago
Major Version Upgrade Aurora PostgreSQL 11asked 2 years ago
Does Neptune execute bulk loader jobs in parallel or serially?asked 2 years ago