- Newest
- Most votes
- Most comments
The high ReadIOPs you're experiencing during your database migration to RDS Aurora PostgreSQL via AWS Database Migration Service (DMS) are likely due to the collection of statistics by DMS. These statistics, including table row counts, are gathered to provide insights into the migration process.
To address this issue during large operations or to disable it permanently, you have a few options:
-
For temporary disabling during large operations:
- You can modify the DMS task settings to disable statistics collection. Look for the "EnableStatistics" parameter in your task settings and set it to "false".
-
For a more permanent solution:
- If you're using the AWS console, you can modify the task and go to the "Task settings" section. Look for "Enable CloudWatch logs" and "Enable statistics" options and uncheck them.
- If you're using the AWS CLI or API, you can modify these settings in the task configuration JSON.
-
Another approach to reduce the impact:
- Instead of completely disabling statistics, you can adjust the frequency of statistics collection. Look for the "StatisticsCollectionInterval" parameter in your task settings and increase its value to reduce the frequency of statistics gathering.
-
At the database level:
- You can consider adjusting the
default_statistics_target
parameter in your PostgreSQL configuration. Lowering this value can reduce the amount of statistics gathered by PostgreSQL itself, which might help in reducing the overall impact.
- You can consider adjusting the
Remember that disabling statistics completely may impact your ability to monitor the progress and performance of your migration tasks. It's often a balance between reducing the impact on your database and maintaining visibility into the migration process.
After making these changes, monitor your ReadIOPs and pg_stat_activity to see if the situation improves. If you're still experiencing issues, you may need to further optimize your DMS task configuration or consider breaking your migration into smaller batches.
Sources
AWS Database Migration Service - Oracle to Aurora MySQL Migration Playbook
PostgreSQL Endpoint Troubleshooting - AWS Database Migration Service
State where row counts are published.
This does not apply to Homogeneous DMS.
Relevant content
- asked 7 months ago
- AWS OFFICIALUpdated 3 years ago
This is "Convert and Migrate", aka Homogeneous DMS. Noted autovacuums started running while tables were still being initialized/COPY'd. Recommend disabling target's autovacuum before starting migration.