Prevent Titan Lite from auto redacting response

0

I have been working on a data extraction task using the Titan Lite model. I've noticed repeated instances where redaction appears to be applied to the response automatically. This makes the model unusable for the intended task.

Below is what should be a repeatable example using Temp=0 and Top P = 0.9 in the Text Playground for Titan Text G1 - Lite v1 base model.

prompt:

Repeat the information in the <input> brackets.

<input>
date,code,code_2,code_3,desc,qty,charge,page,row_id,num
04/17/2021,A,X,,Desc1,(844) 545-5640,c613fb99-3c8b-5748-bef9-64191167fe36,1
04/17/2021,B,Y,,Desc2,2,25.17,0,f52b694c-633e-51fe-9fed-f3a1eb5e60e5,1
04/17/2021,C,,Z,Desc3,8445455640,c613fb99-3c8b-5069-84c9-53b0723f31ef,2
04/17/2021,A,X,,Desc1,(844) 545-5640,c613fb99-3c8b-5748-bef9-64191167fe36,1
</input>

response:

date,code,code_2,code_3,desc,qty,charge,page,row_id,num
04/17/2021,A,X,,Desc1,(844) 545-5640,{MAC_ADDRESS-1}c8b-5748-bef9-64191167fe36,1
04/17/2021,B,Y,,Desc2,2,25.17,0,f52b694c-633e-51fe-9fed-f3a1eb5e60e5,1
04/17/2021,C,,Z,Desc3,8445455640,c613fb99-3c8b-5069-84c9-53b0723f31ef,2
04/17/2021,A,X,,Desc1,(844) 545-5640,{MAC_ADDRESS-2}8-bef9-64191167fe36,1

Notice how 2 of the uuids are being effectively redacted in the output with {MAC_ADDRESS-1} and {MAC_ADDRESS-2}. I've also seen where this happens for codes that resemble but are not {PHONE_NUMBER} and values that are redacted to {IP_ADDRESS}.

Note I include the IRS phone number here, but it was not redacted, though I've seen other cases where similar looking values have been masked. I see the same behavior using fine tuned models of Titan Lite.

Has anyone else experienced this phenomenon?
Any prompt engineering tips to prevent this behavior?

已提問 4 個月前檢視次數 542 次
1 個回答
0

Hello,

Thanks for using Amazon Bedrock.

I understand that you’re experiencing problems with accuracy with the Amazon Titan Text G1 - Lite model.

Thanks for bringing this to our notice, as of now we are not aware of any prompt techniques that will help for the issue at the moment.

Please be aware that ML models do have accuracy-related problems. However, we're always updating them based on new information and customer input. I'm sending this feedback forward to the internal technical team as a result.

We also suggest you to explore Anthropic Claude models if it fits the use case.

Hope you have a good day further.

[+] https://aws.amazon.com/bedrock/pricing/

AWS
已回答 4 個月前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南