1 Answer
- Newest
- Most votes
- Most comments
2
Yes, you can create multiple Glue jobs in the same CDK stack file. To create another Glue job "pos table replication large" scheduled at 9am daily, you can follow these steps:
- Create a new Glue job similar to the existing one:
const L2gluejobLarge = new GlueJob(this, 'pos_rep_job_large', { jobName: 'pos table replication large', workerType: 'G.2X', numberOfWorkers: 20, timeout: Duration.hours(6), glueVersion: '4.0', connections: [pos_gluedbconnection], script: GlueScript.fromAsset(this, 'pos_table_replication_large', './pos Table Replication Large.py'), outputbucket: GlueBucketPath, outputbucketname: GlueBucket });
- Create a new trigger for the new job:
const glue_trigger_L2glue_job_large = new aws_glue.CfnTrigger( this, "glue_trigger_pos_glue_job_large", { name: "Run-Job-pos-table-replication-large", workflowName: glue_workflow.name, actions: [ { jobName: 'pos table replication large', }, ], type: "SCHEDULED", schedule: 'cron(0 9 * * ? *)', // Run at 9am daily startOnCreation: true, } ); glue_trigger_L2glue_job_large.node.addDependency(L2gluejobLarge); glue_trigger_L2glue_job_large.node.addDependency(GlueBucket);
This approach allows you to create multiple Glue jobs in the same CDK stack, each with its own schedule. The new job will use the same crawler and JDBC database connection as the existing job, but it will have its own schedule to run at 9am daily. Make sure to adjust the script path and any other specific properties for the new job as needed.
Sources
@aws-cdk/aws-glue-alpha module · AWS CDK
class CfnTrigger (construct) · AWS CDK
answered 12 days ago
Relevant content
- asked 4 months ago
- asked 3 years ago
- asked a year ago
- AWS OFFICIALUpdated 3 years ago
- AWS OFFICIALUpdated 3 years ago
- AWS OFFICIALUpdated a year ago