- Newest
- Most votes
- Most comments
When using AWS Database Migration Service (DMS) with PostgreSQL as the source for a large-scale migration involving hundreds of schemas, you can expect some impact on the source system. Here's what you should know and consider:
- Resource Impact:
For the initial full load:
- DMS performs full table scans on the source, which can increase I/O and CPU usage.
- This phase may have a more noticeable impact on the source system's resources.
For ongoing CDC (Change Data Capture):
- DMS uses PostgreSQL's native logical replication feature, which requires specific configuration.
- It will lead to some additional I/O and CPU usage as DMS reads from the transaction logs.
- There may be increased storage requirements to retain transaction logs long enough for DMS to process them.
- Performance Implications:
- In most cases, the impact on the source database's performance is minimal, especially during CDC.
- However, during peak times or for very busy systems, you might notice a slight increase in resource usage.
- The initial full load phase is likely to have more impact than the ongoing CDC phase.
- Best Practices to Minimize Impact:
- Schedule the initial full load during off-peak hours if possible.
- Monitor your source database's performance closely during the migration process.
- Ensure your source system has adequate resources to handle both normal operations and the additional DMS workload.
- Consider adjusting PostgreSQL parameters like max_logical_replication_workers and max_sync_workers_per_subscription to optimize performance during CDC.
- For large tables, consider dividing them into separate tasks using selection rules. This can help manage the load on the source system.
- If you find that you're overburdening your source database, reduce the number of tasks or tables for each task in your migration.
- Configuration Considerations:
- Enable row-based binary logging for PostgreSQL to support CDC.
- Set up logical replication slots for PostgreSQL, which is required for CDC.
- Ensure you have the necessary permissions on the source database for DMS operations.
- Testing:
- Before proceeding with the production migration, it's crucial to perform a test migration in a non-production environment that closely mimics your production setup. This will help you assess the actual impact on your specific database configuration and workload.
By carefully planning your migration, monitoring performance, and following these best practices, you can minimize the impact on your source PostgreSQL database while ensuring a successful migration of your business-critical schemas.
Sources
Will DMS task have any impact on the source database? | AWS re:Post
Migrating data from PostgreSQL databases with homogeneous data migrations in AWS DMS - AWS Database Migration Service
Usually DMS with PostgreSQL has always been something to look after, especially when migrating from Oracle. As I understood here it's a migration from PostgreSQL.
What I can say is that depending on the custom attunements of the current schemas on the SOURCE DB there may be several factors to cure about that may bring DMS to an halt or non performing status.
Document every customization you have and every non standard workflow and check on AWS Documentation whether this is supported by DMS and target DB.
Relevant content
- asked a year ago
- asked a year ago
- asked 3 years ago