- Newest
- Most votes
- Most comments
Hello rePost-User-7244213, I understand you're experiencing an issue with running a simple Glue job script in the Ireland region, while the same script runs without problems in the Virginia region. This seems to be a recent issue, as you mentioned complex scripts were working fine yesterday. Let's address this problem step by step:
- Verify your IAM permissions: Ensure that your IAM user or role has the necessary permissions to run Glue jobs in the Ireland region. Sometimes, permissions can be region-specific.
- Check Glue service status: Visit the AWS Service Health Dashboard to see if there are any reported issues with AWS Glue in the Ireland region.
- Try a different Glue job version: If you're using a specific Glue version, try changing it to a different one (e.g., from Glue 3.0 to Glue 2.0 or vice versa).
- Validate your script: Although your script is simple, double-check for any syntax errors or unexpected characters that might have been introduced.
- Check your Glue job configuration: Review the job parameters, ensuring that all required fields are correctly filled and there are no misconfigurations.
- Attempt to run the job using AWS CLI: Try running the job using the AWS CLI to see if you get more detailed error information:
aws glue start-job-run --job-name YourJobName --region eu-west-1 - Monitor CloudWatch Logs: Check the CloudWatch Logs for your Glue job to see if there are any additional error messages or stack traces that could provide more insight.
- Try a different script: Create a new Glue job with a different simple script to see if the issue persists.
- Test in a different account: If possible, try running the same job in a different AWS account to determine if it's an account-specific issue.
If the problem continues after trying these steps, it's recommended to contact AWS Support. They can investigate potential region-specific issues or account-level problems that might not be visible to you.
To verify if the problem has been resolved:
- Successfully run your simple script in the Ireland region.
- Monitor the job execution in the AWS Glue console and verify that it completes without errors.
- Check the job's output in the designated S3 bucket (if applicable) to ensure it produced the expected results.
Remember, if you're using the AWS free tier, be aware of the service limits and ensure you haven't exceeded any quotas, as this could potentially impact job execution.
Additional Sources:
AWS Glue machine learning exceptions - AWS Glue
Troubleshooting blueprint errors in AWS Glue - AWS Glue
Hopefully this helps give some direction on solving the issue. All the best and thank you for using AWS!
Brian
Relevant content
- AWS OFFICIALUpdated 10 months ago
can you share some more job details and whats in the log? Do you see InvalidInputException in your logs? Is the job a Spark or Python Glue job?
In the log I also see: failed to execute with exception Task allocated capacity exceeded limit. (Service: AWSGlueJobExecutor; Status Code: 400; Error Code: InvalidInputException; Request ID: XXX; Proxy: null). This job is a Spark job.
I have the exact same error in the Ireland region (eu-west-1).
In my case I am using an AWS Lake Formation Blueprint which creates a Workflow and some Jobs in AWS Glue. I have run it multiple times and the workflow always fails with this error.
I'm having the same issue for pythonshell job type in Ireland region, I haven't tested other regions yet. The only way to correct the problem that I've found is destroying the glue job and, then, re-creating it again. Does anybody has any other kind of solution for this problem?