Nova Lite Batch Inference Error: Document Format

Question

I am trying to do batch inference with Nova Lite to identify possible OCR errors. I tried to attach a CSV document as part of the content; however, I'm getting the following error:

> Malformed input request: #/messages/0/content/0/document/format: csv is not a valid enum value, please reformat your input and try again.

Unfortunately, this error occurred after Validation, so the job showed as "Completed." From what I see in the documentation, CSV is one of the document formats that is supposed to be supported:

https://docs.aws.amazon.com/nova/latest/userguide/modalities-document.html
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/bedrock-runtime/client/converse.html

Below is an example of the JSON I am sending for each request. Is a formatting error causing this, or am I missing something about what document formats are supported? The portions in capital letters are variables passed in, and the actual JSONL input files are using one line per request.

```
{
    "recordId": RECORD_ID,
    "modelInput": {
        "system": [
            {
                "text": SYSTEM_PROMPT
            }
        ],
        "messages": [
            {
                "role": "user",
                "content": [
                    {
                        "document": {
                            "format": "csv",
                            "name": "csv",
                            "source": {
                                "bytes": CSV_IN_BASE64_STRING
                            }
                        }
                    },
                    {
                        "text": USER_PROMPT
                    }
                ]
            },
            {
                "role": "assistant",
                "content": [
                    {
                        "text": "```json"
                    }
                ]
            }
        ]
    }
}
```

Malte Reimann · Answer

Hello,

The Amazon Bedrock batch inference API support model input in the format of the InvokeModel API.

> The format of the modelInput JSON object must match the body field for the model that you use in the InvokeModel request. 
https://docs.aws.amazon.com/bedrock/latest/userguide/batch-inference-data.html

InvokeModel request bodies can't include documents.

> You can't include a document in the request body. To include a document during inference, use the Chat/text playground in the AWS Management Console or send a Converse request.
https://docs.aws.amazon.com/bedrock/latest/userguide/inference-invoke.html

re:Post Agent · Answer

It appears that you're encountering an issue with the document format specification in your batch inference request for Amazon Nova Lite. While CSV is indeed one of the supported document formats for Amazon Nova models, the error message suggests that there might be a discrepancy between the expected input format for batch inference and the format you're using.

For batch inference with Amazon Bedrock models, including Nova Lite, the input format is typically more streamlined compared to the conversational API format you're using. The batch inference input usually doesn't include the full conversation structure with system prompts and multiple messages.

Instead, try simplifying your input to focus on the essential elements needed for inference. Here's a suggested format for your batch inference input:

```json
{
    "recordId": "RECORD_ID",
    "modelInput": {
        "prompt": "YOUR_PROMPT_HERE",
        "document": {
            "format": "csv",
            "source": {
                "bytes": "CSV_IN_BASE64_STRING"
            }
        }
    }
}
```

In this format:
1. Include the "recordId" if you want to track individual records.
2. Under "modelInput", provide a "prompt" field with your instruction or question.
3. Include the "document" object with the "format" and "source" as you had before.

This simplified structure should be more in line with what the batch inference endpoint expects. If you need to include system instructions or additional context, consider incorporating them into the prompt text.

If you continue to experience issues after adjusting your input format, it may be worth double-checking a few other points:
1. Ensure that the base64 encoding of your CSV is correct.
2. Verify that the CSV content adheres to the size limits (should not exceed 4.5MB per document).
3. Confirm that you're not exceeding the maximum of 5 documents per request.

If the problem persists after these adjustments, it would be advisable to contact AWS Support for more specific guidance, as there might be nuances to the Nova Lite batch inference process that aren't fully documented or that have changed recently.  
**Sources**  
[Document understanding - Amazon Nova](https://docs.aws.amazon.com/nova/latest/userguide/modalities-document.html)  
[Error in AWS Bedrock Batch Inteference | AWS re:Post](https://repost.aws/questions/QUBN3gF6eNSqSpniExiexuMQ/error-in-aws-bedrock-batch-inteference)  
[Can you specify custom data in the input bucket's jsonl records when doing batch inferencing? | AWS re:Post](https://repost.aws/questions/QUoFPlzGW0Tqir6Xf4f28vrg/can-you-specify-custom-data-in-the-input-bucket-s-jsonl-records-when-doing-batch-inferencing)

Nova Lite Batch Inference Error: Document Format

Relevant content