Pass parameter from one glue job to another in step function?

When you configure a Step Function to orchestrate AWS Glue jobs, you can use the output of one job as input to another. Here's how you can achieve it:

• AWS Glue Job 1: This job normalizes the data in the file and writes it to an Iceberg table. You should modify this job to return the date information that you want to pass to the next job. This can be done by writing the date information to a location where the second job can access it, such as an S3 bucket or by using job parameters.
• Step Function State Output: The Step Function state that runs the first AWS Glue job should be configured to capture the output of the Glue job. This can be done by setting the appropriate state output configuration in the Step Function definition. The output can include the location of the stored date information or the date itself.
• Pass the Output to the Next State: You can then pass this output to the next state in the Step Function. The Step Function state that invokes the second Glue job should be configured to use the output of the first state as its input.
• AWS Glue Job 2: The second Glue job should be configured to read the input passed from the previous state. This input will contain the date information you need. The job can then proceed to read the Iceberg table and write one file per date.

If this has resolved your issue or was helpful, accepting the answer would be greatly appreciated. Thank you!