How to improve Comprehend results when the model returns incorrect results?

Question

What is the best way to improve the results I receive back from Comprehend when using Custom Entities?  
  
I've trained the model, but the results are still falling short.  I'm not sure if training the model with more annotation examples will improve my results.  It would be great to be able to correct the model when it gets something wrong.  What is the best way to accomplish my goal of improving the output over time?  
  
For example,  AWS returned the value of _"select common flexbase_" as one single "_material_" type.  This should have been three separate "_material_" types (and was annotated hundreds, if not thousands, of times separately in the training annotations).    
  
In another example from the same sample email, AWS returned the value of "_Rowlett @ Bickers Drive & Westover Drive Export Trees Trash Concrete Asphalt Dirt Import Select Common Flexbase (Crushed Conc) Flexbase (Crushed Limestone)......._" as one single "_address_" type.  The address part was only "_Rowlett @ Bickers Drive & Westover Drive_" with the majority of the remainder being individual "_material_" types.  
  
I have a lot of other examples, but it's a bit difficult explaining them without the benefit of attaching the source files and the AWS Comprehend output.

Answer

Hi,  
   I am from Comprehend Team and sent you private message for requesting dataset. We can analyse the model and error samples to suggest next steps.  
  
Thanks

Answer

Hi,  
  
May I know, if I follow up your suggestion, one document as one line, after training and gets the result of F1 score, how can I know which line has the problem in the detail?  
  
Thanks.  
  
Best regards

Answer

If anyone else is struggling with this I was able to achieve better results in my training dataset by having **each document be a single line** in the training file (rather than observing line breaks in the source document).  
  
Example - use a 10 line email as a source document, make the 10 line email one single line (no line breaks) in your training file, not 10 different lines.  
  
I hope this makes sense.

How to improve Comprehend results when the model returns incorrect results?

Relevant content