Comprehend Medical Context Window limits

According to the Amazon Comprehend Medical documentation, the maximum document size for the DetectEntities, DetectEntities-v2, and DetectPHI operations is 20 KB, which is equivalent to 20,000 UTF-8 characters. For context window limits in question/answer scenarios, Amazon Comprehend Medical evaluates the unit per request, where 1 unit equals 100 characters. We can calculate the context window limits based on the number of tokens and characters.

Based on the OpenAI General FAQ, 1 token is approximately equivalent to 4 characters in English. According to IBM's documentation on UTF Unicode, each character is encoded as 1 to 4 bytes in UTF-8, with the first 128 Unicode code points encoded as 1 byte. For simplicity, we will assume 1 byte per character.

Therefore, the maximum document size of 20 KB is equivalent to 20,000 bytes or 20,000 characters. To determine the number of tokens, we divide 20,000 characters by 4, resulting in 5,000 tokens.

Amazon Comprehend Medical also allows 100 Transactions per Second (TPS) for the DetectEntities-v2, DetectEntities, DetectPHI, InferRxNorm, and InferICD10CM operations. This means you can process up to 2 million characters per second and 500,000 tokens per second in these operations.

In summary:

• Maximum document size: 5,000 tokens or 20,000 characters
• Throughput: 500,000 tokens per second or 2 million characters per second in specific operations