If you merely want to copy the full database to a new host, and then change some of the data values and such, you may be best served by using
mysqldump and dumping the source database, then importing it to the new host, where you run a query to make the transformations and update the values you want. this task?
From a cost and performance perspective, I would suggest you follow an approach where "everything" happens within the host. Serverless models like AWS Glue have a pay-per-use billing model where you pay for each byte of data that is processed. Moving data between RDS, EC2 and S3 would also involve data transfer cost.
If I were you, I would:
- Create a Stored Procedure in db1 that does the data transformation and load the data in db2.
- If the data load needs to happen instantly when data changes in db1, then write triggers in source tables to invoke the stored procedure.
- If the data needs to be updated periodically, then find a way to invoke the stored procedure at fixed intervals. For example: a) using an ETL package running on an EC2 instance or b) a Lambda function (triggered by an EventBridge Schedueler) that executes the stored procedure.
- asked 5 months ago
- asked a month ago
- Why does my AWS Glue job fail with lost nodes when I migrate a large data set from Amazon RDS to Amazon S3?AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated 6 months ago
- AWS OFFICIALUpdated 8 days ago
- AWS OFFICIALUpdated a year ago
- EXPERTpublished a month ago
- EXPERTpublished 5 days ago