Discrepancy in Fine-Tuning Billing - Token Count Issue Titan Text G1 - Express

Thank you for sharing your concern about the discrepancy in token count for your fine-tuning job with Titan Text G1 - Express. There are a few important points to consider regarding this issue:

1. Token count vs. file size: It's important to note that the number of tokens doesn't directly correlate with the file size in KB or MB. Tokenization is a process that breaks down text into smaller units (tokens), which can result in a higher token count than the raw character count.

2. Token limit for fine-tuning: For Titan Text G1 - Express v1 fine-tuning jobs, there is a maximum limit of 10,000 combined training and validation records. This limit is in place to ensure efficient processing and prevent overuse of resources.

3. Repeated processing: If your training data exceeded this limit, it's possible that the system attempted to process it multiple times, which could lead to a significantly higher token count than expected.

4. Tokenization ratio: While it's true that typically each character doesn't translate to more than one token, the exact tokenization depends on the specific tokenizer used by the model. Some words or phrases might be tokenized into multiple tokens.

Given the large discrepancy between your file size and the reported token usage, it does seem unusual and warrants further investigation. Here are some steps you can take:

1. Review your training data to ensure it doesn't exceed the 10,000 record limit.
2. Check if there were any unintended repetitions or loops in your fine-tuning job configuration.
3. Verify if there were multiple fine-tuning attempts or jobs that might have contributed to the total token count.

It's good that you've already opened a support ticket. AWS support should be able to provide a detailed breakdown of the token usage and help identify any potential issues with your fine-tuning job. In the meantime, you might want to pause any ongoing fine-tuning jobs to prevent further unexpected charges.

For future fine-tuning jobs, it's advisable to start with a smaller dataset to get a better understanding of how the tokenization and billing work for your specific use case. This can help you estimate costs more accurately and avoid unexpected charges.
Sources
Amazon Bedrock endpoints and quotas - AWS General Reference
Amazon Titan Text models - Amazon Bedrock

Hi there,

I'm so very sorry to hear that your support case is taking a long time to be resolved.

I can understand your need to have this looked into. I've forwarded the concerns that you've shared here, directly to our team working your case for additional visibility.

Due to this platform being unauthenticated, I'm unable to discuss case or account details here.

However, our Support team has the needed details to assist you further. Please continue keeping an eye on your case correspondence for further updates from our team.

— Aimee K.