Sagemaker Ground Truth completes with labelling errors

0

Hi all,

I set a Sagemaker Ground Truth labelling task. All the labelled objects results with "Failed" status, though it is possible retrieve the labels from S3. Eventually, nothing apart from the job status is wrong - but I am worried that I could and I would like to resolve it. There are no logs in cloudwatch with anything FAILED. Would you know how to get rid of this issue? For start, it would be good to see a specific issue in cloudwatch and then try to resolve the problem with failed jobs.

The logs show the following (I filtered out repeating events for different input objects):

{
    "labeling-job-name": "test-job51",
    "event-name": "INPUT_REQUEST_VALIDATED",
    "event-log-message": "Input request validated."
}
{
    "labeling-job-name": "test-job51",
    "event-name": "CREATED_WORKER_INTERFACE",
    "event-log-message": "Human task UI created."
}
{
    "labeling-job-name": "test-job51",
    "event-name": "INPUT_MANIFEST_PARSED",
    "event-log-message": "Input manifest parsed successfully."
}
{
    "labeling-job-name": "test-job51",
    "event-name": "PRE_HUMAN_LAMBDA_INVOKED",
    "event-log-message": "Pre-human task Lambda invoked for line 6."
}
{
    "labeling-job-name": "test-job51",
    "event-name": "PRE_HUMAN_LAMBDA_SUCCEEDED",
    "event-log-message": "Pre-human task Lambda returned successfully for line 6."
}
{
    "labeling-job-name": "test-job51",
    "event-name": "HUMAN_TASK_CREATED",
    "event-log-message": "Human task created successfully for line 6."
}
{
    "event-name": "HUMAN_TASK_COMPLETED",
    "event-log-message": "Human task completed successfully for line 6.",
    "labeling-job-name": "test-job51"
}
{
    "labeling-job-name": "test-job51",
    "event-name": "LABELING_JOB_PROGRESS",
    "event-log-message": "Labeling job progress for batch number:1 | #Total data set objects:2 | #Data set objects completed:0 | #Data set objects waiting for humans:0 | #Data set objects failed:2 | #Data set objects expired:0 | #Data set objects cancelled:0"
}

The setup goes as follows

  • using VPC
  • using Private Labelling team
  • CORS set on S3 bucket (per suggestion from https://repost.aws/knowledge-center/sagemaker-ground-truth-errors)
  • the execution role has permission to use cloudwatch (per suggestion from https://repost.aws/knowledge-center/sagemaker-ground-truth-errors)
  • Cognito App callback url is https://XXX.labeling.us-east-1.sagemaker.aws/oauth2/idpresponse and logout URL is https://XXX.labeling.us-east-1.sagemaker.aws/logout (I noticed that when the labelling job/workforce is created from Console, there is an additional allowed callback https://XXX.cloudfront.net; I did not add it)
adam
asked a year ago126 views
No Answers

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions