AWS Sagemaker Compilation Job Not Submitting

0

Hello!

Was attempting to create/run a Sagemaker Compilation job yesterday and it would not submit. Pressing the button did not generate any errors and all required fields were seemingly filled. Pressing the button left me on the page with all the fields still full. The network page in the browser dev console didn't reveal any errors.

I have all the necessary permissions and the quotas are good on the account. Any suggestions?

asked 2 months ago32 views
1 Answer
0

If you're experiencing issues with submitting a SageMaker Compilation job without any visible errors, there are a few things you can check:

  1. CloudWatch Logs: Even if no errors are displayed in the console, there might be underlying issues logged in CloudWatch. Navigate to the CloudWatch console, look for the log group "/aws/sagemaker/CompilationJobs", and check for any error messages or clues about why the job isn't submitting.

  2. IAM Permissions: Although you mentioned having the necessary permissions, it's worth double-checking. Ensure your IAM role has the correct permissions to create and run compilation jobs, including access to the required S3 buckets and SageMaker resources.

  3. Service Quotas: While you've stated that the quotas are good on the account, it's possible there might be a specific quota related to compilation jobs that's been reached. Review your SageMaker service quotas in the AWS console to confirm there are no limitations for compilation jobs.

  4. Region-specific issues: Sometimes, services can have temporary issues in specific regions. Try creating the compilation job in a different AWS region to see if the problem persists.

  5. Input data and model: Verify that all input data, including the model file, is correctly uploaded to S3 and accessible. Ensure the model format is compatible with SageMaker Neo compilation.

  6. Browser issues: Clear your browser cache and cookies, or try using a different browser to rule out any client-side issues.

  7. AWS Support: If none of the above steps resolve the issue, it might be worth contacting AWS Support. They can look into account-specific details or any known service issues that might be affecting your ability to submit compilation jobs.

Remember, even if the console doesn't show an error, there might be underlying issues preventing the job submission. Checking CloudWatch logs and ensuring all prerequisites are met are crucial steps in troubleshooting this kind of silent failure.
Sources
Troubleshoot Neo Compilation Errors - Amazon SageMaker
Troubleshooting guide - Amazon SageMaker AI

profile picture
answered 2 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions