Use GroundTruth bbox labels in a TensorFlow fine tuning job

0

I can use the manifest file created by Ground Truth ref as input to a training job for object detection (ref). Can also use it for Tensorflow jobs, like the one here? In all the tutorials I find, the data in annotations.json has a different format than the one in the GroundTruth output.

My goal is to use more ad-hoc models rather than just resnet and vgg, and get information from Tensorboard and such.

Lorenzo
질문됨 일 년 전230회 조회
1개 답변
1
수락된 답변

Hello Lorenzo,

In general, you can use Ground Truth augmented manifest to train a TensorFlow model with SageMaker using Pipe Mode. Augmented manifests can only support Pipe input mode.

So the specific example you asked here is not going to work using augmented manifest, because it's script mode. If you just meant to use the estimator, you can modify the code to something like this.

Please have a read on and follow this for more details.

AWS
지원 엔지니어
Jann_P
답변함 일 년 전
profile picture
전문가
검토됨 한 달 전
  • Thanks Jann! What I would like to do is to train an object detector model using images annotated with Ground Truth. The model could be YOLO V5 or similar. I would like to have access to Tensorboard for metrics visualization. Do you have code examples to do that? Thanks!

  • Hello Lorenzo, Yes, you can follow the document below to set up the Tensorboard. Please note this will create an app in your domain so cost will occur.

    "Amazon SageMaker with TensorBoard runs the TensorBoard application on an ml.r5.large instance and incurs charges after the SageMaker free tier or the free trial period of the feature. For more information, see Amazon SageMaker Pricing."

    https://docs.aws.amazon.com/sagemaker/latest/dg/tensorboard-on-sagemaker.html

  • Thanks Jann. The problem seems to be that the Jumpstart models (e.g. in s3://jumpstart-cache-prod-us-east-1/source-directory-tarballs/tensorflow/transfer_learning/od1/v1.1.0/sourcedir.tar.gz) don't know how to open those annotation files it seems. I'm seeing the error:

    FileNotFoundError: [Errno 2] No such file or directory: '/opt/ml/input/data/training/annotations.json'

    The jumpstart code is still looking for the annotation.json file, and it probably wants to use that to load the annotations

  • In general, would the models in Jumpstart be able to load the images and annotations listed in the augmented manifest file produced by GroundTruth?

  • Unfortunately, it is not supported for Jumpstart models as they are expecting a different type of format. You may need to, which you probably have done, write a custom script to convert the format.

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠