RDS Read Replica Lag

Question

I have an db.m5.8xlarge RDS MySQL database in the us-east-2 region with 15k Max IOPS and a read replica of the same size and specifications in us-east-1.

When our business day starts our replica lag starts. It generally reached about 6000-8000 second on any given business day.

While this is a high transaction database, this level of lag makes using the read replica for anything other than a straight back up impossible.

I have spent hours researching and trying to tune the MySQL settings to get this replica lag under control.

Does anyone else have any experience solving this problem?

I am running out of ideas.

Thank you.

Answer

Have you looked into configuring parallel apply on the read replica? By default MySQL replication uses single threaded apply. So if your workload on the source database uses high concurrent connections, these can be serialized on the read replica unless you configure parallel apply.
All workloads do not necessarily tend to parallelize as well as others so your mileage might vary, but that is something you can try.

https://dev.mysql.com/doc/refman/8.0/en/replication-options-replica.html
see: replica-parallel-workers

Answer

Dear Customer,

Thanks for reaching out to AWS re:Post
Adding on the discussions already on the post, I wanted to add a public doc (in case you have missed it previously)->

https://aws.amazon.com/premiumsupport/knowledge-center/rds-mysql-high-replica-lag/#:~:text=Amazon%20RDS%20for%20MySQL%20uses,you%20can%20monitor%20replication%20lag.

Hope this helps in the project

RDS Read Replica Lag

Relevant content