1 réponse
- Le plus récent
- Le plus de votes
- La plupart des commentaires
0
You can run any script in script runner mode.
https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-hadoop-script.html
Basically from the step function invoke the syntax thats required by the script runner, for example:
"StartAt":"Step 1",
"States":{
"Step_1":{
"Type":"Task",
"Resource":"arn:aws:states:::elasticmapreduce:addStep.sync",
"Parameters":{
"ClusterId.$":"$.ClusterId",
"Step":{
"Name":"1 - Step 1",
"ActionOnFailure":"CONTINUE",
"HadoopJarStep":{
"Jar":"s3://elasticmapreduce/libs/script-runner/script-runner.jar",
"Args":[
"s3://xxx/scripts/step1.sh"
]
}
}
},
"End":true
}
}
}
Put all the code in the step1.sh script, this script will execute on master node and do any task you want including your sqoop stuff
Contenus pertinents
- Réponse acceptéedemandé il y a un an
- demandé il y a un an
- demandé il y a 7 mois
- AWS OFFICIELA mis à jour il y a un an
- AWS OFFICIELA mis à jour il y a 3 ans