Skip to content

API Gateway call - S3 put object using VTL

0

I wanted to create AWS API Gateway which integrates to S3 put object using Integration type as AWS Service S3 Put using VTL in integration mapping to create the unique S3 Path

3 Answers
4

Hi ,

please go through the below steps i hope it will helps to resolve your issue.

Create an S3 Bucket:

  • Log in to your AWS Management Console.
  • Go to the S3 service and create a new bucket if you don't already have one.

Create a New API in API Gateway:

  • Go to the API Gateway service.
  • Click "Create API" and choose "HTTP API".

Set Up the API Method:

  • Create a new resource or select an existing one.
  • Create a new method (e.g., PUT).

Configure the Integration Request:

  • Choose "Integration type" as "AWS Service".
  • Select "AWS Service" as "S3".
  • Set the HTTP method to "PUT".
  • Set "Action Type" to "Use path override" and specify the path to your S3 bucket, such as /bucket-name/{object}.

Set Up VTL Mapping Templates:

  • In the Integration Request, set up a mapping template for the content type you are using (e.g., application/json).
  • Use VTL to generate a unique S3 path. Here is an example of a VTL mapping template that creates a unique path based on a timestamp:
#set($context.requestOverride.header.x-amz-meta-custommetadata = $input.params('metadata'))
#set($bucketName = "your-bucket-name")
#set($objectKey = $input.params('object'))
{
    "bucket": "$bucketName",
    "key": "$context.requestTimeEpoch/$objectKey",
    "x-amz-meta-custommetadata": "$context.requestOverride.header.x-amz-meta-custommetadata",
    "body": $input.body
}

Configure Integration Response:

  • Set up integration responses to map S3 responses back to your API responses.
  • Configure the necessary status codes and response mappings.

Deploy the API:

  • Deploy your API to a stage (e.g., prod).
  • Make sure to note the endpoint URL for testing.

Set Up IAM Role for API Gateway to Access S3:

  • Create an IAM role with the necessary permissions for API Gateway to put objects in your S3 bucket.

Attach a policy to this role like:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::your-bucket-name/*"
        }
    ]
}
  • In the API Gateway method settings, specify this role under "Execution role".

Test the API:

  • Use a tool like Postman or curl to send a PUT request to your API endpoint with the necessary payload.
  • Verify that the object is created in the specified S3 bucket with a unique path.

Please go through below documentation once

https://docs.aws.amazon.com/apigateway/latest/developerguide/integrating-api-with-aws-services-s3.html

https://aws.amazon.com/blogs/compute/patterns-for-building-an-api-to-upload-files-to-amazon-s3/

EXPERT
answered 2 years ago
EXPERT
reviewed 2 years ago
2

Hello,

Please try this solution it will be helpful for you.

To create an API Gateway that integrates with S3 to put an object using VTL, follow these steps. First, create an S3 bucket and an IAM role with a policy allowing s3:PutObject actions on your bucket. Then, in the API Gateway console, create a new REST API, add a resource (e.g., /upload), and create a PUT method. Set the integration type to AWS Service, select S3, and use path override /your-bucket-name/{object}. In the Integration Request, set up a mapping template for application/json with the following VTL script:

#set($context.requestOverride.header.Authorization = "")
#set($context.requestOverride.header.Content-Type = "application/octet-stream")
#set($context.requestOverride.header.x-amz-acl = "public-read")

{
    "bucket": "your-bucket-name",
    "key": "$input.params('object')",
    "content": "$util.base64Decode($input.body)"
}

Replace your-bucket-name with your actual bucket name. Configure Integration Response mappings as needed and deploy your API. You can then test the API using a tool like Postman or curl, ensuring to replace placeholders like your-api-id and base64-encoded-content with actual values. This setup allows you to upload objects to your S3 bucket via API Gateway with VTL for request mapping.

Please look at AWS Document Link you will get more information.

https://docs.aws.amazon.com/apigateway/latest/developerguide/integrating-api-with-aws-services-s3.html

https://aws.amazon.com/blogs/compute/patterns-for-building-an-api-to-upload-files-to-amazon-s3/

EXPERT
answered 2 years ago
0

Thank you for your response. I am getting below error.

{Object} is not getting transformed from mapping template.

Execution failed: URI/URL syntax error encountered in signing process Thu Jun 13 13:03:45 UTC 2024 : Endpoint request URI: https://s3.us-east-1.amazonaws.com/my-bucket/{object} Thu Jun 13 13:03:45 UTC 2024 : Endpoint request headers: {x-amz-meta-custommetadata=, x-amzn-apigateway-api-id=tbfswhb50c, Accept=application/json, User-Agent=AmazonAPIGateway_tbfswhb50c, X-Amzn-Trace-Id=Root=1-666aee31-2f656dd0d25165416a7a20c3, Content-Type=application/json} Thu Jun 13 13:03:45 UTC 2024 : Endpoint request body after transformations: { "bucket": "my-bucket", "key": "1718283825220/", "x-amz-meta-custommetadata": "", "body": {"data":"hello","PartitionKey":"1"} } Sending request to https://s3.us-east-1.amazonaws.com/my-bucket-name/{object} Thu Jun 13 13:03:45 UTC 2024 : Execution failed due to configuration error: Illegal character in path at index 69: https://s3.us-east-1.amazonaws.com/my-bucket-name/{object}

body:

{"data":"hello","PartitionKey":"1"}

Method: post

Thu Jun 13 12:43:54 UTC 2024 : Execution failed due to configuration error: Illegal

answered 2 years ago
  • Hi @Ahmed Abu Bakar, Were you able to resolve this issue? actually i am also facing the same issue, if you can help me understand what could be the root cause of this issue?

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.