Llama 3 providing empty response and string response instead of JSON

Question

I am using Llama 3 70B through AWS Bedrock SDK. I observe that 90%+ responses are not in JSON, as mentioned in the prompt. Instead, I'm getting blank responses or string responses. Can anyone help me with the issue?

When I check the same on playground, I never faced any issue.

rePost-User-9561832 · Accepted Answer

Hi there,

I understand that you are using Llama 3 70B through AWS Bedrock SDK and you have observed that you are getting blank responses.

Can you check if prompt format follows the guidelines provided for Llama 3 models.

Details about the tags on the prompt can be found here
[1] https://llama.meta.com/docs/model-cards-and-prompt-formats/meta-llama-3/

If the issue persists and you're unable to determine the cause, consider reaching out to AWS support for assistance. They can investigate further and provide guidance based on the specifics of your situation.

Llama 3 providing empty response and string response instead of JSON

Relevant content