Amazon Rekognition experienced a service issue - Why is this error happening?

0

Hello,

I have tried to train an Amazon Rekognition (custom label) object detection model on a set of labeled medical images to detect medical tools. I have about 23,000 labeled images with 8 classes. I have tried training the model 2 times, but both times I am getting a "training failed" message after almost 24 hours of training: "Amazon Rekognition experienced a service issue." How can I solve this? Is there any reason this error is occurring?

The model seemed to successfully train when I used a smaller subset of the data with only about 1500 labeled images. Is the dataset size the issue?

Any help would be appreciated! Thank you.

4개 답변
1

Hello AWS-User-0102979, thanks for using Amazon Rekognition Custom Labels. Our team has noticed these training jobs that failed due to service issues in the last few days, and is currently investigating what is the root cause. AWS Rekognition CustomLabels supports upto 250,00 images for Object Detection/Localization. Please refer to here: https://docs.aws.amazon.com/rekognition/latest/customlabels-dg/limits.html . Please consider opening a case via AWS Support Center. As son as we identify the problem, we will get back to you. Thanks for reaching out.

Thanks, aws-yasarka

답변함 2년 전
0

Hello AWS-User-0102979, Can you please share the AWS Region in which you ran these trainings and approximate time of trainings to help us debug the failures.

답변함 2년 전
  • I am not exactly sure what AWS region these trainings were done, but it may have been the US East (N. Virginia) based on my location.

    the last failed trainings were done on:

    2022-02-07 T21.16.02 February 07, 2022 2022-02-06 T09.51.06 February 06, 2022 2022-02-04 T18.29.17 February 04, 2022

0

Please confirm if you region is US West 2 (PDX/Oregon). You can sign-in to you AWS Account Web Console, and you can see the region info on the top right corner. If you are using aws-cli, you can get region info in your AWS config file. We have been informed with multiple training failures between 02/04/2022 and 02/06/2022. We will let you know when we identify and resolve the issue as soon as possible.

AWS
답변함 2년 전
0

Hello, Can you please try training again ?

AWS
답변함 2년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠