By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Deploy Multiple Glue Jobs(100+) using CodeDeploy and Not Time dependent.

0

Hi Team,

I have a query -- How can I do multiple deployments of glue jobs(100+) with different scripts from same pipeline and assign them different Job names?

I have made a CFT for cross-account deployment using Codepipeline, and I'm able to successfully deploy a single glue job. But for the next phase in our project, we need to deploy multiple glue jobs with different scripts and they are not time-dependent.

Like a developer pushed his code on 7th Feb, and another developer on 18th Feb like this there can be 100+ such jobs and each job must have different names. And also, deployment of another glue job should not affect the previous ones unless there is a change in the Native script.

So, how can I achieve this type of deployment?

  • knamburi
    a year ago

    Have you considered AWS Cloud Development Kit (AWS CDK) for your deployment needs? Using CDK, you could create reusable CDK construct libraries using common programming languages. This approach would be ideal in your case with multiple developers committing code.

  • Garvit Tiwari
    a year ago

    Hey @knamburi, haven't considered it yet!! Can you share some DOCs for this?

    I have my cft for cross-pipeline ready.

  • knamburi
    a year ago

    Hi Garvit,

    1. Please find CDK AWS developer guide here. This will help you get started : https://docs.aws.amazon.com/cdk/v2/guide/home.html.

    2. Sample code is available for your reference that helps accelerate your IAC pipeline https://github.com/aws-samples/aws-glue-cdk-cicd

Topics
Developer ToolsDevOps
Tags
AWS CodeBuildAWS CodePipelineAWS CodeCommitAWS CodeDeployDevOps
Language
English
Garvit Tiwari
asked a year ago103 views
No Answers

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content