Run multiple Datasync jobs in parallel

0
Run multiple Datasync jobs in parallel
I have an agent installed in my data center that is located in a private network. I can start one job at a time to move the data from on prem to s3. If I started another job it's queued (waiting for the first job to be completed). I believe this is not the way it should work. Is there something that is not well configured? Can anyone help in this? I want to know If I'm allowed to run several jobs to upload the data using the same agent from my data center to s3
posta 5 mesi fa392 visualizzazioni
1 Risposta
0

Hi!

Remember the following about running multiple tasks:

  1. Tasks run in a series (first in, first out). If you run the same task more than once, by default each execution goes into a queue. If you start a task twice, for instance, the second task execution won't start until the first one finishes.
  2. You can run separate tasks at the same time, but these tasks will be queued if they're using the same DataSync agent.

You can also take a look at this document: https://aws.amazon.com/es/blogs/storage/how-to-accelerate-your-data-transfers-with-aws-datasync-scale-out-architectures/

You may need to set multiple agents to run the same task in parallel Enter image description here

con risposta 5 mesi fa

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande