AWS StepFunctions [HTTP Task]: when RequestBody + ResponseBody = 256kb does not go into Catch Task.

Question

im using the StepFunction HTTP Task with a Catch Task attached. When the RequestBody **exceeds** the 256kb limit it goes into the Catch Task just fine.

However, when the RequestBody is e.g. 130kb and the ResponseBody again has 130kb it exceeds the 256kb limit and hence I assume it also goes into the attached Catch Task.
That's sadly not the truth and the Http Task gets stuck in a ** Canceled** state and throws a States.DataLimitExceeded error and does **NOT** go into the Catch Task.

```
{
  "Next": "webhook-state-sqs-task-dev",
  "Type": "Task",
  "Resource": "arn:aws:states:::http:invoke",
  "Parameters": {
    "ApiEndpoint.$": "$.endpoint",
    "Method.$": "$.webhook_config.httpMethod",
    "QueryParameters.$": "$.queries",
    "Authentication": {
      "ConnectionArn.$": "$.webhook_config.connectionArn"
    },
    "RequestBody.$": "$.payload"
  },
  "ResultPath": "$.http_response",
  "Catch": [
    {
      "ErrorEquals": [
        "States.ALL"
      ],
      "ResultPath": "$.error",
      "Next": "webhook-task-catch-error-dev"
    }
  ]
}
```

imho this is a bug. If not please at least document it and explain why.
Thanks a lot!

rePost-User-6429935 · Accepted Answer

Hello,

To begin with your problem statement, I can see that you are encountering a "States.DataLimitExceeded" error with the total size of the RequestBody and ResponseBody for the HTTP Task, which is the causing an error that is not being caught by your Catch Task as expected.

AWS Step Functions, States.DataLimitExceeded exception occurs under the following conditions:

- If the output from a connector exceeds the payload size quota.
 - If the output of a state is larger than payload size quota.
 -  If, after Parameters processing, the input of a state is larger than the payload size quota.

It is clearly mentioned in the below documentation that States.ALL error type can't catch the States.DataLimitExceeded terminal error type and runtime error types. So, it cannot be caught by the Catch Task designed for States.DataLimitExceeded.

[+] https://docs.aws.amazon.com/step-functions/latest/dg/concepts-error-handling.html#error-handling-error-representation

Payload Size Limits :
AWS Step Functions enforces a payload size limit of 256 KB. This limit applies to the combined size of both input and output data for tasks such as Lambda functions, activities, or integrated services.
For more details, check this [documentation](https://docs.aws.amazon.com/step-functions/latest/dg/limits-overview.html#service-limits-task-executions) to review Quotas.

How to Resolve the Issue "States.DataLimitExceeded" error, consider these approaches:

- Optimize Payload Size:  Reducing the size of the RequestBody and ResponseBody payloads by filtering or compressing the data.
Filter Data: Review and reduce the amount of data being passed. Remove any unnecessary information.
Compress Data: Utilize compression techniques to decrease the data size.

- Leverage Amazon S3: To store large payloads
Store Large Payloads: For large data payloads, use Amazon S3. Store your data in an S3 bucket and pass the S3 object key or ARN to your Step Functions execution. This way, the actual payload size does not affect the Step Functions execution.
Retrieve Data from S3: Lambda functions or other service tasks can then read from or write to S3 as needed.

[+]Amazon S3 ARNs -  https://docs.aws.amazon.com/step-functions/latest/dg/avoid-exec-failures.html

In case you have additional queries or follow-up queries or if the problem still persists, don't hesitate to reach out to AWS Support. We may require details that are non-public information to assist you better. Please open a support case with AWS using the following [link](https://support.console.aws.amazon.com/support/home#/case/create).

AWS StepFunctions [HTTP Task]: when RequestBody + ResponseBody >= 256kb does not go into Catch Task.

Relevant content