Fine Tuning Validation error: At least 16 examples needed for each train/eval file.

0

I am following AMZ Workshop : 02_fine-tuning_llama2.ipynb to try to fine-tune the the cohere.command-light-text-v14:7:4k model. They are using the CNN 3.0.0 dataset and prepping it (see Prepare CNN news...) so that it looks like:

training data:

{ "prompt": "Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n\ninstruction:\n\nSummarize the news article provided below.\n\ninput:\n\n(article snippet)", "completion": "response:\n\n(example summary)" }

The validation data is looks like:

{"prompt": "Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n\ninstruction:\n\nSummarize the news article provided below.\n\ninput:\n\n(article snippet)", "completion": "response:\n\n(example response)"}

The training file contains 5k rows, and the validation file contains 1k rows. This works with the amazon.titan-text-lite-v1:0:4k model. However, using the cohere command light model I get:

Validation error: At least 16 examples needed for each train/eval file. train file doesn't have enough examples.

I checked AWS Prepare training and validation datasets..., but the format looks the same.

Any idea why it is giving the "Validation error" or how to fix it?

This Bedrock fine tuning ValidationError question is similar, but I don't think the suggestions fix the issue I'm having.

1 Answer
0

Hello,

I understand that you are seeing a Validation error when trying to fine tune the 'cohere.command-light-text-v14:7:4k' model. However, this works with the 'amazon.titan-text-lite-v1:0:4k' model. As per the error, please ensure to check the following-

  1. Please refer to the documentation [1] for the supported hyperparameters for model customization of Cohere Command and Cohere Command Light models.
  2. Kindly refer to the documentation [2][3] for more information about fine tuning Cohere models and dataset details.

You can also refer to this sample blog/workflow [4][5] to fine tune the Cohere Command Light model.

Kindly try checking the above resources. If you face any issues or require further assistance, please reach out to AWS Support [6], and we would be happy to assist you further. Thank you!

References:

[1] https://docs.aws.amazon.com/bedrock/latest/userguide/cm-hp-cohere-command.html

[2] https://docs.cohere.com/docs/fine-tuning#how-to-create-fine-tuned-models

[3] https://docs.cohere.com/docs/datasets

[4] https://serverlessland.com/workflows/bedrock-model-customisation

[5] https://github.com/aws-samples/step-functions-workflows-collection/tree/main/stepfunctions-bedrock-model-customisation-sam

[6] https://console.aws.amazon.com/support/home#/case/create

AWS
answered a month ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions