Uploading greater than 6 MB file to S3 through API Gateway results in Request too long error, Is that expected ?

Question

As I read in AWS documentation, the payload limitation for the API Gateway is 10 MB. My Requirement is to upload files directly to S3 through API Gateway.

When I upload file of greater than 6 MB size, I get "Request too long" error and CloudWatch log shows payload size exceeds 10 MB limit . But while uploading files as binary which is less than 6 MB, it is working as expected. Also, in my case I cannot use Pre-signed URL as an option.

Please suggest any alternative to upload files above 6 MB.

Answer

Because binary files are encoded for upload (usually Base64) that increases the size of them during transfer which is probably what you're seeing.

Instead, consider having an API call which triggers a Lambda function to return a pre-signed URL to the client which will allow upload directly to S3. That way you won't have to deal with the limit in API Gateway.

Some example Python code for the Lambda function:

```
s3 = boto3.client('s3')
presignedURL = s3.generate_presigned_url('put_object',
                                         Params={'Bucket':s3BucketName, 
                                                 'Key':s3ObjectKey',
                                                 'ContentType':objectContentType}
                                         ExpiresIn=linkExpiryTimeInSeconds)
```

Uploading greater than 6 MB file to S3 through API Gateway results in Request too long error, Is that expected ?

Relevant questions