Issue with large inputs into Bedrock Prompt Flow

Question

I recreate a tool I built in AWS and Bedrock. I am pulling text from an s3 into my flow to add to prompt which is in Prompt Management. When I am using a smaller test s3 file, it works. When I use an actually representative example of the s3 file I'm going to pull, I get Internal Server Exception error. I need to be able to add this level of detail to the prompt, plus users my add additional context. Is this a timeout thing? Is it an input limit things? (I'm using 3.5 sonnet). Is there any way around these limitations, or should I just scrap AWS and Bedrock for this use case? Is there a way to change these configurations like there are in lambda functions? Are there other options to utilize large prompts with these tools? In this scenario, chunking or streaming isn't an option

Vitor Castellani · Answer

Hi Matt.

The issue you're facing with large inputs into Bedrock Prompt Flow is likely due to input size limits or other quota restrictions, such as maximum input size or processing time. Bedrock enforces strict quotas that aren't adjustable, which could cause the internal server errors you're experiencing when working with large S3 files.

To work around this, you could pre-process or reduce the input size before feeding it into the prompt flow. If this isn't feasible, you might need to explore alternative services better suited for handling large data inputs.

For further details, refer to the [AWS documentation on quotas](https://docs.aws.amazon.com/bedrock/latest/userguide/quotas.html) and [Prompt Flows](https://docs.aws.amazon.com/bedrock/latest/userguide/flows.html).

Matt · Answer

Thank you for your reply. For reference, the input tokens tend t average from around 4000  with no additional context added, to around 6-7k, if the user adds context files. The output is generally 600-1000. (It's another app I built that we wanted to try recreating with AWS). Does this seem like it should cause those timeouts?

Issue with large inputs into Bedrock Prompt Flow

Relevant content