Improve slow DMS table validation query

0

DMS is using this query to perform table validation post full load migration (in this case on a MySQL source) :

SELECT tableID FROM ( SELECT @row:=@row+1 AS rownum, tableID FROM (SELECT @row:=0) v, ( SELECT tableID FROM mySchema.bigTable ORDER BY tableID ASC ) AS sorted ) as ranked WHERE rownum % [PartitionSize] = [PartitionNumber]

It is quite slow on a table with over 10 million rows (20 seconds for each 10000 partition pass) or painfully slow on a table with over 50 million rows (around 7 minutes for each 10000 partition pass).

Why not use this much faster query until no rows are returned?

MySQL:

SELECT tableID FROM mySchema.bigTable ORDER BY tableID ASC LIMIT [PartitionNumber]*[PartitionSize],[PartitionSize]

Oracle / PostgreSQL:

SELECT tableID FROM mySchema.bigTable ORDER BY tableID ASC LIMIT [PartitionSize] OFFSET [PartitionNumber]*[PartitionSize]

SQL Server:

SELECT tableID FROM mySchema.bigTable ORDER BY tableID ASC OFFSET [PartitionNumber]*[PartitionSize] ROWS FETCH FIRST [PartitionSize] ROWS ONLY

profile picture
已提问 1 年前99 查看次数
没有答案

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则