Ground Truth - Active Learning Status

Question

I currently have a Ground Truth labeling job in progress for which I enabled the active learning feature. I see the activelearning folder in the output directory within s3, but all that's in there right now is a file called active_learning_info.json. The job is on its 4th batch of images, which at 250 for each batch, means it is at almost 1,000 objects labeled by Mechanical Turk workers at this point. Based on that as well as reading https://docs.aws.amazon.com/sagemaker/latest/dg/sms-data-output.html and https://aws.amazon.com/blogs/machine-learning/annotate-data-for-less-with-amazon-sagemaker-ground-truth-and-automated-data-labeling/ , I was expecting to see additional contents within the activelearning folder. Is that not accurate? How do I know if active learning is truly being (at least) attempted within this labeling job?  
  
Thank you!  
  
Edited by: naquent on May 16, 2019 1:05 PM

Answer

Answering my own question: I checked again this morning, and now I see all of the expected folders and files. I guess it just takes a while for it to show up. In this case, the lowest numbered folder in the activelearning folder is 8. My guess is that this is corresponding to the 8th batch, which would mean it took 8 batches of 250 images for it to start attempting active learning. Is that the correct understanding? Either way, I'll mark this thread as answered.

Answer

Apologize for the delay in the reply.  
That is correct. The lowest number of the folder would be when we started to "attempt" active learning. You should see input/output files that might throw some light on to which data objects were tried in those batches. But, this does not necessarily mean that it got auto annotated. The best way to check the objects that got auto annotated is the final output file where it would be marked by a metadata field "human-annotated":"no" after the job is done. Or, you can check the folder "intermediate" where we store the output of each batch.   
  
We hope this answers your question. Thank you for being a valued customer.

Ground Truth - Active Learning Status

Relevant content