I created a Glue job in AWS Glue Studio and connected it to GitLab using a Personal Access Token (PAT). ## Behavior - Push to repository works successfully (Glue generates and commits the JSON file) - Pull from repository fails with the error: Unable to pull job JOBNAME from Gitlab REPO/BRANCH.. Unable to convert the input to Job model. ## Details - The JSON in GitLab is generated by AWS Glue via "Push to repository" - No manual changes are made before attempting Pull - The issue is consistently reproducible ## Question Why would Glue fail to pull a job definition that it generated itself? Are there undocumented constraints on the JSON format or known issues with GitLab integration?

Thank you for the response. In our case, the repository is **not under a GitLab Group**. It is a **user namespace**, so the documented limitation regarding GitLab Groups should not apply. Additional clarifications: - We are **not modifying the JSON** in any way. The file is generated directly by AWS Glue via "Push to repository" and remains unchanged before attempting to pull. - The JSON file is successfully created in the repository, which suggests that: - authentication is working - repository path and folder configuration are correct - The Personal Access Token includes the following scopes: - api - read_api - read_repository - write_repository - Since **Push works successfully**, it indicates that: - the token is valid - permissions are sufficient - repository access is correctly configured However, **Pull consistently fails** with: Unable to convert the input to Job model Given that Glue cannot pull back a job definition that it generated itself, this appears to be a potential issue on the service side rather than a configuration problem. Could you please confirm if there are any known issues or undocumented constraints related to GitLab integration or JSON parsing during `Pull from repository`? Thank you.

Based on the available information about AWS Glue's Git integration, there is a known limitation that may be relevant to your issue: AWS Glue currently does not support pushing/pulling from GitLab-Groups. If your repository is organized under a GitLab Group structure, this could explain why the pull operation fails even though the push succeeds. However, regarding your specific error "Unable to convert the input to Job model," this suggests a JSON parsing or validation issue that isn't explicitly documented in the standard AWS Glue Git integration documentation. While AWS Glue does support GitLab as a provider (along with GitHub, Bitbucket, and AWS CodeCommit), the error you're experiencing where Glue cannot parse its own generated JSON is not a documented known issue. Some potential areas to investigate: - Verify that your GitLab repository is not organized under a GitLab Group, as this configuration is not supported - Ensure the repository structure matches AWS Glue's expectations (correct folder paths and file naming) - Confirm that the authentication strategy and permissions are correctly configured for both push and pull operations - Check that the job definition file hasn't been inadvertently modified by GitLab's auto-formatting or any repository hooks Since this appears to be an edge case where Glue fails to parse its own output, you may need to engage AWS Support to investigate whether there's a service-side issue or undocumented constraint affecting the JSON serialization/deserialization process between push and pull operations. **Sources** [Using Git version control systems in AWS Glue - AWS Glue](https://docs.aws.amazon.com/glue/latest/dg/edit-job-add-source-control-integration.html) [update_job_from_source_control - botocore 1.42.50 documentation](https://docs.aws.amazon.com/botocore/latest/reference/services/glue/client/update_job_from_source_control.html)

Glue Version Control Pull fails from GitLab with an error Unable to convert the input to Job model

I created a Glue job in AWS Glue Studio and connected it to GitLab using a Personal Access Token (PAT).

Behavior

Push to repository works successfully (Glue generates and commits the JSON file)
Pull from repository fails with the error: Unable to pull job JOBNAME from Gitlab REPO/BRANCH.. Unable to convert the input to Job model.

Details

The JSON in GitLab is generated by AWS Glue via "Push to repository"
No manual changes are made before attempting Pull
The issue is consistently reproducible

Question

Why would Glue fail to pull a job definition that it generated itself? Are there undocumented constraints on the JSON format or known issues with GitLab integration?

Topics: Analytics
Tags: AWS Glue
Language: English

Tanya

asked a month ago38 views

2 Answers

Newest
Most votes
Most comments

This recommendation was generated using AWS Generative AI capabilities. You are responsible for evaluating the recommendation in your specific context and implementing appropriate oversight and safeguards. Learn more

Based on the available information about AWS Glue's Git integration, there is a known limitation that may be relevant to your issue: AWS Glue currently does not support pushing/pulling from GitLab-Groups. If your repository is organized under a GitLab Group structure, this could explain why the pull operation fails even though the push succeeds.

However, regarding your specific error "Unable to convert the input to Job model," this suggests a JSON parsing or validation issue that isn't explicitly documented in the standard AWS Glue Git integration documentation. While AWS Glue does support GitLab as a provider (along with GitHub, Bitbucket, and AWS CodeCommit), the error you're experiencing where Glue cannot parse its own generated JSON is not a documented known issue.

Some potential areas to investigate:

Verify that your GitLab repository is not organized under a GitLab Group, as this configuration is not supported
Ensure the repository structure matches AWS Glue's expectations (correct folder paths and file naming)
Confirm that the authentication strategy and permissions are correctly configured for both push and pull operations
Check that the job definition file hasn't been inadvertently modified by GitLab's auto-formatting or any repository hooks

Since this appears to be an edge case where Glue fails to parse its own output, you may need to engage AWS Support to investigate whether there's a service-side issue or undocumented constraint affecting the JSON serialization/deserialization process between push and pull operations.
Sources
Using Git version control systems in AWS Glue - AWS Glue
update_job_from_source_control - botocore 1.42.50 documentation

re:Post Agent

answered a month ago

EXPERT

Oleksii Bebych

reviewed a month ago

Thank you for the response.

In our case, the repository is not under a GitLab Group. It is a user namespace, so the documented limitation regarding GitLab Groups should not apply.

Additional clarifications:

We are not modifying the JSON in any way.
The file is generated directly by AWS Glue via "Push to repository" and remains unchanged before attempting to pull.
The JSON file is successfully created in the repository, which suggests that:
- authentication is working
- repository path and folder configuration are correct
The Personal Access Token includes the following scopes:
- api
- read_api
- read_repository
- write_repository
Since Push works successfully, it indicates that:
- the token is valid
- permissions are sufficient
- repository access is correctly configured

However, Pull consistently fails with:

Unable to convert the input to Job model

Given that Glue cannot pull back a job definition that it generated itself, this appears to be a potential issue on the service side rather than a configuration problem.

Could you please confirm if there are any known issues or undocumented constraints related to GitLab integration or JSON parsing during Pull from repository?

Thank you.