Textract Custom Queries Adapter suddenly started to hallucinate values

0

We trained an adapter based on 500 documents and it worked perfectly for weeks until I noticed something strange today.

Some queries started returning multiple values for the same query without appearing in the detected raw text.

For example: For the query "What is the license plate number?" the adapter would previously only output one value which would be "ABC123". Now, it adds a second answer with the exact same confidence value, which is either a duplicate of the correct answer or complete gibberish/made up.

In some instances, the read out values do not even appear in the raw text and they don't have any bounding boxes attached, Textract simply seems to hallucinate.

I was able to replicate this issue on every document I tried to analyze using the adapter.

This is a huge issue to say the least. I don't know what happened. As we did not change anything within the code or on the adapter version, so I'm assuming, that it might be an issue with the Textract model.

Any chance I could fall back on a previous version of Textract? Are you aware of this issue?

Thanks in advance.

Huseyin
asked 4 months ago194 views
1 Answer
0

Hi, Thanks for reaching out. There was no recent update to the Textract Custom Queries model. So we would like to understand the behavior here - 1/ Did you retrain the adapter before you noticed the change in behavior? , 2/ Are these new document types/layouts that are being tested on the adapter? Can you submit a support ticket, so we can reach out to you and deep dive?

AWS
SM
answered 4 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions