Fine Tune LLM - Anthropic Claude model - Bedrock

Hello.

As stated in re:Post Agent's answer, it cannot be used as a PDF, so it needs to be converted to JSONL format.
Conversion is required to the format described in the document below.
https://docs.aws.amazon.com/bedrock/latest/userguide/model-customization-prepare.html#model-customization-dataset-prep

{"system": "<system message>","messages":[{"role": "user", "content": "<user query>"},{"role": "assistant", "content": "<expected generated text>"}]}

While the article you mentioned specifically discusses fine-tuning the Claude 3 Haiku model using JSONL formatted data, it is not directly possible to fine-tune the model using PDF documents in their raw format. However, you can still use PDF documents as a source for fine-tuning by following these steps:

1. Extract the text content from your PDF documents.
2. Process and structure the extracted text into the required JSONL format for fine-tuning.
3. Ensure that each line in your JSONL file contains a system prompt and a message, as required by the fine-tuning process for Claude 3 Haiku.

The fine-tuning process in Amazon Bedrock requires the training data to be in a specific JSONL format, with each line containing a system prompt and a message. The total size of the training data should not exceed 10GB, and the validation data should not exceed 1GB.

To use PDF documents for fine-tuning, you would need to develop a preprocessing pipeline that:

1. Extracts text from PDFs
2. Structures the extracted information appropriately
3. Formats the data into the required JSONL structure

This approach allows you to leverage the content of your PDF documents for fine-tuning while adhering to the format requirements of the Claude 3 Haiku model in Amazon Bedrock. Remember that the quality and relevance of the extracted and formatted data will significantly impact the success of your fine-tuning process.
Sources
Fine-tuning for Anthropic’s Claude 3 Haiku model in Amazon Bedrock is now generally available | AWS News Blog
Best practices and lessons for fine-tuning Anthropic’s Claude 3 Haiku on Amazon Bedrock | AWS Machine Learning Blog