By using AWS re:Post, you agree to the Terms of Use
/AWS Data Pipeline stuck in "WAITING FOR RUNNER"/

AWS Data Pipeline stuck in "WAITING FOR RUNNER"

0

Hi,

all my AWS Data Pipelines i create to export data from a dynamodb table to a .csv in an s3 bucket are stuck in the "WAITING FOR RUNNER" state. I am following the tutorial here: https://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-importexport-ddb.html

I also checked the values "Runs on resource" which is: "EmrClusterForBackup" and "workerGroup" which is "df-0274180EXA6BQJAA9HV_@EmrClusterForBackup_2019-01-13T10:21:37"

can you tell me if these values are correct?

asked 3 years ago79 views
4 Answers
0

Common issue would be ,

  1. check the IAM access of emr ec2 role and make sure that it has datapipeline:*
  2. make sure that internet connectivity is available from emr/ec2 either nat or igw

you can add ec2 key in emr cluster and log in to emr master (ssh) then check task runner log

tail -100f /mnt/taskrunner/output/logs/tasrunner*.log

https://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-how-task-runner-user-managed.html#dp-verify-task-runner

answered 3 years ago
0

Hi Shivan,
thank you for your answer.

I think with EMR EC2 Role you mean the DataPipelineDefaultResourceRole? The one i assign durin the creation of the pipeline to the field "EC2 instance role"?
If so, i checked. It has datapipeline full access.

If i activate my datapipeline, then switch to the EMR console, and check the respective cluster, it says "Install Task Runner cancelled".

Do you have an idea what the issue could be?

Best regards

Edited by: OlliHC on Jan 14, 2019 7:08 AM

Edited by: OlliHC on Jan 14, 2019 7:14 AM

answered 3 years ago
0

check your pipeline terminateaftertimeout parameter and increase 2-3 times normal run +15 minutes for cluster launch

Example,
Job usual run time is 10 minutes then 30 +15 =45 minutes would be best value for terminateafter

answered 3 years ago
0

Just to be precise:
I fixed it.

The reason was that our limit on EC2 instances of the respective account was reached.
We texted the AWS support and their increased our limit, now it works fine.

answered 3 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions