DMS Controlling the number of source database connections used

Question

Is this possible?  
  
I have to stagger my ETL processes out because DMS opens up ~34(+7regular) connections on our Postgres source database to complete one transfer task.  The table count is 224 for this task/database (one example, there are many databases).  If I don't stagger them the total connection count goes way above 100 and then our source server essentially crashes.  It simply won't handle that many open connections at once.   
  
Is there any way to control the total connections AWS DMS uses per task job?    
  
Thanks  
  
Edited by: ted1111 on Sep 30, 2021 8:06 AM

Answer

Actually by adjusting a couple settings in the task JSON downward I was able to reduce the total resources required on the source database.  
  
See the MaxFullLoadSubTasks and CommitRate settings.  
  
For MaxFullLoadSubTasks it generally corresponds to the number of tables processing at once, although number connections is multiples per table in my case.  For CommitRate I lower this to 1000 to conserve all resources on our side, server and network, reduce overall stress.  
  
Edited by: ted1111 on Oct 1, 2021 3:21 AM  
  
Edited by: ted1111 on Oct 1, 2021 9:53 AM

DMS Controlling the number of source database connections used

Contenuto pertinente