Job bookmark causes to load no data into my job

Question

My AWS Glue job keeps failing on daily basis because there is no data to be processed which is not true. I have a system in place which adds new data every morning at 1AM to my S3. Then I use Job to read in those tables. When I disable my bookmark then it works with no issue, but I don't want to re process all data every day.

I have a job init in place, job commit, transformation context in my tables so that should be fine. Any idea what might be wrong?

```
job = Job(icebergContext.glueContext)
job.init(args["JOB_NAME"], args)

if target in ['meta', 'all']:
    meta = ProcessMeta(database, target_database, icebergContext, "adform_meta")
    meta.process_meta()

#then inside of the process_meta here is an example of table load:
labels = self._icebergContext.getDataFrameFromCatalog(database=self._database, table_name="adform_v2_labels_v2", transformation_ctx='adform_labels')
job.commit()

```
Any kind of feedback is appreciated!

re:Post Agent · Answer

The issue you're experiencing with your AWS Glue job and job bookmarks is likely related to how the job is interacting with the new data in your S3 bucket. Here are a few potential reasons and solutions to consider:

1. Job Bookmark Implementation: Ensure that you're using the AWS Glue DynamicFrame API to read data from your source, not the DataFrame API or Spark SQL. The DynamicFrame API is required for job bookmarks to function correctly.

2. Transformation Context: While you mentioned having a transformation context in place, double-check that you're using a unique identifier for each ETL operator instance. This is crucial for job bookmarks to track processed data correctly.

3. S3 Input Source: For S3 input sources, job bookmarks use the last modified time of objects to determine which ones need processing. If your new data is being added to existing files rather than creating new files, the job bookmark might not detect the changes.

4. Data Source Property: Ensure you haven't changed the data source property (like the S3 input path) while the bookmark is enabled. Changing the input path without changing the transformation context can lead to missing or skipping files.

5. Bookmark Reset: If you've recently deleted and recreated the table using a crawler, this could have reset the job bookmark, causing it to process all data again.

6. Concurrent Runs: Make sure you've set the maximum number of concurrent runs for the job to 1. This helps prevent issues with job bookmarks.

7. JDBC Source Considerations: If you're using a JDBC source, ensure that the columns used as bookmark keys are properly sorted and don't have gaps.

To troubleshoot further, you could try the following:

1. Review your job logs to see if there are any error messages or indications of why no data is being processed.

2. Temporarily enable more verbose logging to get more insight into the job's behavior.

3. Verify that the new data in your S3 bucket is indeed in a location and format that your Glue job expects.

4. Consider implementing a custom bookmark solution if the built-in job bookmark feature doesn't suit your specific use case.

By carefully reviewing these aspects of your Glue job configuration and data pipeline, you should be able to identify and resolve the issue causing your job to process no data while bookmarks are enabled.  
**Sources**  
[AWS Job Bookmark not working and not creating temp files | AWS re:Post](https://repost.aws/questions/QUHlFmTwgzTpefV3KxklaboA/aws-job-bookmark-not-working-and-not-creating-temp-files)  
[Troubleshoot AWS Glue job bookmarks | AWS re:Post](https://repost.aws/knowledge-center/glue-job-bookmarks)  
[glue job bookmark | AWS re:Post](https://repost.aws/questions/QUDsm-30oRTG2ZAZSL8VF8Lw/glue-job-bookmark)  
[Tracking processed data using job bookmarks - AWS Glue](https://docs.aws.amazon.com/glue/latest/dg/monitor-continuations.html)

Job bookmark causes to load no data into my job

Add your answer

Relevant content