Amazon Rekognition experienced a service issue - Why is this error happening?

0

Hello,

I have tried to train an Amazon Rekognition (custom label) object detection model on a set of labeled medical images to detect medical tools. I have about 23,000 labeled images with 8 classes. I have tried training the model 2 times, but both times I am getting a "training failed" message after almost 24 hours of training: "Amazon Rekognition experienced a service issue." How can I solve this? Is there any reason this error is occurring?

The model seemed to successfully train when I used a smaller subset of the data with only about 1500 labeled images. Is the dataset size the issue?

Any help would be appreciated! Thank you.

4 回答
1

Hello AWS-User-0102979, thanks for using Amazon Rekognition Custom Labels. Our team has noticed these training jobs that failed due to service issues in the last few days, and is currently investigating what is the root cause. AWS Rekognition CustomLabels supports upto 250,00 images for Object Detection/Localization. Please refer to here: https://docs.aws.amazon.com/rekognition/latest/customlabels-dg/limits.html . Please consider opening a case via AWS Support Center. As son as we identify the problem, we will get back to you. Thanks for reaching out.

Thanks, aws-yasarka

已回答 2 年前
0

Hello AWS-User-0102979, Can you please share the AWS Region in which you ran these trainings and approximate time of trainings to help us debug the failures.

已回答 2 年前
  • I am not exactly sure what AWS region these trainings were done, but it may have been the US East (N. Virginia) based on my location.

    the last failed trainings were done on:

    2022-02-07 T21.16.02 February 07, 2022 2022-02-06 T09.51.06 February 06, 2022 2022-02-04 T18.29.17 February 04, 2022

0

Please confirm if you region is US West 2 (PDX/Oregon). You can sign-in to you AWS Account Web Console, and you can see the region info on the top right corner. If you are using aws-cli, you can get region info in your AWS config file. We have been informed with multiple training failures between 02/04/2022 and 02/06/2022. We will let you know when we identify and resolve the issue as soon as possible.

AWS
已回答 2 年前
0

Hello, Can you please try training again ?

AWS
已回答 2 年前

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则