Introduction

Batch processing is an important component of enterprise applications running on mainframe, and as these batch processes are migrated from mainframe to AWS, they require similar integration between batch processing and scheduling functions.

If the scheduler requirements are simple, such as triggering the head of the job stream at specific schedule and submitting the subsequent jobs in series or in parallel, then Amazon EventBridge and AWS Step Functions can be utilized in combination to build the scheduler. AWS Step Functions defines the job flow and Amazon EventBridge scheduler triggers the job flow at a specific time.

Architecture Diagram

Below architecture shows how a single job is triggered at a specific schedule by Amazon EventBridge scheduler. The Job poller pattern within AWS Step Functions orchestrates the Starting, Monitoring and Reporting success/failure of the job by making API calls to the AWS Mainframe Modernization service.

Create a job template

A job template is created using the AWS Step Functions' Job Poller pattern with parameterized option to submit any batch script.

Use the job template to create job flow

Below diagram shows a sample job flow using the AWS Step Functions JobTemplate created earlier. Jobs S1, S2, Parallel Set and S3 run serially. Jobs P1, P2 and P3 run in parallel. The job flow itself is triggered by Amazon EventBridge.

For complete working solution, refer to the github repo.

References

[1] Submit batch jobs for AWS Mainframe Modernization applications

[2] AWS Mainframe Modernization API Reference

[3] Guidance for Scheduling Batch Jobs for AWS Mainframe Modernization