How do you pass special parameters for AWS Glue jobs via AWS CloudFormation

0

When creating a AWS Glue ETL Job with AWS CloudFormation, how do you specify advanced options such as additional JAR's that the job may require, special security configuration parameters for KMS encryption, etc?

AWS
asked 5 years ago1809 views
1 Answer
0
Accepted Answer

Hi,

In AWS CloudFormation when using the Resource AWS::Glue::Job under Properties you need to add these advanced parameter in the DefaultArguments section.

            "DefaultArguments": {
            "--TempDir": "s3://aws-glue-temporary-633748029103-eu-west-1/serhat",
            "--class": "GlueApp",
            "--enable-glue-datacatalog": "",
            "--enable-metrics": "",
            "--encryption-type": "sse-s3",
            "--job-bookmark-option": "job-bookmark-enable",
            "--job-language": "scala"
        },

You can find the list of parameters here.

For additional JARS: --extra-jars is the parameter.

For Server Side Encryption: --encryption-type is the parameter.

Hope this helps.

EXPERT
Serhat
answered 5 years ago
AWS
EXPERT
Raphael
reviewed 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions