The manifest file contains too many invalid rows

0

I am trying to add visuals to the project and dataset I created in rekognition custom label using sdk.

I think there is a problem in my manifest.json file, but I can't figure it out. I followed the solutions for the error at this address but I could not reach a solution:

https://docs.aws.amazon.com/rekognition/latest/customlabels-dg/tm-debugging-aggregate-errors.html

I also use this method:

https://docs.aws.amazon.com/rekognition/latest/customlabels-dg/md-add-images.html

The mistake I made: "errorMessage": "Dataset update failed: UPDATE_FAILED : The manifest file contains too many invalid rows."

The code snippet I am working on is as follows:

    getObjectParams = {
        'Bucket': bucket_name,
        'Key': manifest_file
    }
    
    response = s3Client.get_object(**getObjectParams)
    manifestBody = response['Body'].read().decode('utf-8')
    
    changes=json.loads('{ "GroundTruth" : ' +
            json.dumps(manifestBody) + 
            '}')
    
    responseRek = rekognition.update_dataset_entries(
        Changes= changes, DatasetArn=dataset_arn_train
    )
    print("[INFO] -> Response: ",responseRek)

A piece of my manifest.json file:

[
  {
    "source-ref": "s3://groot20jun23/trainingdata/groot/groot_5.jpg",
    "bounding-box": {
      "image_size": [
        {
          "width": 778,
          "height": 1000,
          "depth": 3
        }
      ],
      "annotations": [
        {
          "class_id": 0,
          "top": 5,
          "left": 60,
          "width": 544,
          "height": 729
        }
      ]
    },
    "bounding-box-metadata": {
      "objects": [
        {
          "confidence": 1
        }
      ],
      "class-map": {
        "0": "groot"
      },
      "type": "groundtruth/object-detection",
      "human-annotated": "yes",
      "creation-date": "2023-06-22T12:22:40.527256",
      "job-name": "myjob"
    }
  },
  {
    "source-ref": "s3://groot20jun23/trainingdata/groot/groot_6.jpg",
    "bounding-box": {
      "image_size": [
        {
          "width": 1440,
          "height": 1958,
          "depth": 3
        }
      ],
      "annotations": [
        {
          "class_id": 0,
          "top": 202,
          "left": 277,
          "width": 708,
          "height": 950
        }
      ]
    },
]

Thank's for help

2 Answers
2

Hi, to locate better your error, I would suggest one to start with a smaller manifest (1 entry ?) to avoid the "too many errors" part of the problem.

Then, when working with 1 entry, you can add more.

If you are sure of the S3 locations, I would think that you may have an authorization error: Rekognition not allowed to access those files. But proving it with a single entry will be most efficient: you will get an explicit message about your error.

Hope it helps!

Didier

profile pictureAWS
EXPERT
answered a year ago
0

Hi, Didier. Your answer solved my problem. When I send my arrayed objects one by one, it adds them to my dataset without any problems. Thank you first of all. But it takes some time to add the images to the dataset. However, the images are on s3 bucket. I wait about 10-11 seconds for 1 image. Do you have any suggestions for this?

Thank's for help.

profile picture
Mustafa
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions