Disable Metrics during Homogeneous Database Migration Service

0

Hello,

While transferring a large database into RDS Aurora Postgresql via DMS, noted ReadIOPs went very high. Reviewing pg_stat_activity, noted several backends constantly running 'select count(*) from' on tables. This is not a trivial operation.

How can this be disabled during large operations? How can this be disabled permanently?

Thanks.

  • This is "Convert and Migrate", aka Homogeneous DMS. Noted autovacuums started running while tables were still being initialized/COPY'd. Recommend disabling target's autovacuum before starting migration.

1 Answer
-1

The high ReadIOPs you're experiencing during your database migration to RDS Aurora PostgreSQL via AWS Database Migration Service (DMS) are likely due to the collection of statistics by DMS. These statistics, including table row counts, are gathered to provide insights into the migration process.

To address this issue during large operations or to disable it permanently, you have a few options:

  1. For temporary disabling during large operations:

    • You can modify the DMS task settings to disable statistics collection. Look for the "EnableStatistics" parameter in your task settings and set it to "false".
  2. For a more permanent solution:

    • If you're using the AWS console, you can modify the task and go to the "Task settings" section. Look for "Enable CloudWatch logs" and "Enable statistics" options and uncheck them.
    • If you're using the AWS CLI or API, you can modify these settings in the task configuration JSON.
  3. Another approach to reduce the impact:

    • Instead of completely disabling statistics, you can adjust the frequency of statistics collection. Look for the "StatisticsCollectionInterval" parameter in your task settings and increase its value to reduce the frequency of statistics gathering.
  4. At the database level:

    • You can consider adjusting the default_statistics_target parameter in your PostgreSQL configuration. Lowering this value can reduce the amount of statistics gathered by PostgreSQL itself, which might help in reducing the overall impact.

Remember that disabling statistics completely may impact your ability to monitor the progress and performance of your migration tasks. It's often a balance between reducing the impact on your database and maintaining visibility into the migration process.

After making these changes, monitor your ReadIOPs and pg_stat_activity to see if the situation improves. If you're still experiencing issues, you may need to further optimize your DMS task configuration or consider breaking your migration into smaller batches.
Sources
AWS Database Migration Service - Oracle to Aurora MySQL Migration Playbook
PostgreSQL Endpoint Troubleshooting - AWS Database Migration Service

profile picture
answered 20 days ago
  • State where row counts are published.

  • This does not apply to Homogeneous DMS.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions