Cloudfront X S3 website + secondary bucket for specific media

Question

Hi everybody, I have a specific need I'm struggling to make it work.

I have a Cloudfront + OAI configured to serve a S3 bucket with a static website. So far so good, the OAI policy in the main bucket is working as expected.

BUT the business requires us to provide some training videos which they don't want to put in the same bucket as the website because it isn't supposed to be static.

I updated the secondary bucket (the one that contains the media) with the same policy as the main bucket (OAI permission) but I'm still getting access denied when trying to load the content.

What is stuck in my head is: If I update the policy to allow access to the secondary bucket from an IP source, the developer can load the content using the S3 URL. If I remove the IP policy and leave only the OAI, the webpage gives 403 for the content. I tried searching the whole CloudFront and the S3 documentation but I can't find a scenario where this happens.

One of the suggestions I saw is to create another CloudFront distribution to serve the secondary bucket but I lack the development knowledge on how to serve the media without creating a whole webpage. Have you ever been through this situation? Are you aware of any other policy I can set up on the secondary bucket to allow my main bucket + OAI to access the media using the S3 URL? I tried using the pre-signed URL solution but I couldn't find a way for the developer to generate and provide this content in the webpage. (I've seen how to do it, I just don't know how to do it taking into consideration the business need).

Thank you for your time and help. I appreciate it.

BazzieB_AWS · Accepted Answer

Hi Gabriel

I work with CloudFront and S3 and I would like to assist you to resolve this issue.

Firstly, your use case is widely used, so there is nothing wrong there.

Generally , S3 returns a 403 when either the incorrect permissions are on the bucket, or you are trying to access a path/file that does not exist. 
Using OAI is also the best way to access S3 bucket, while keeping the bucket private.

So to do this, make sure your settings are as below:
### S3 Bucket
1. Make sure your origin is an S3 origin type, and not a custom origin. This will be for the static bucket and the media bucket. 
2. S3 Bucket Access, select your OAI, and let CloudFront update your bucket policy for you. 
3. Save the changes.

### CloudFront
For the behavior, I am assuming your static site is on the root of the bucket. That is fine.

1. Make sure the media content is in a folder and not the root of the bucket. Example: bucket-name/mediafolder/mediafile.mp4
2. Create a bahavior, set the behavior path = /mediafolder/*
3. For the Origin, select the S3 media bucket.
4. Select your protocol and allowed methods. 
5. If you would like to cache the content, select the CachingOptimized policy, otherwise select the CachingDisabled policy.
6. For the Origin request policy, select the CORS-S3Origin. But if you are not doing CORS requests, then you can leave this as None and save the behavior. 
7. Setup the default behavior for all other requests, mainly for your static bucket.

Please note that when using S3, the URL and behavior path needs to match the S3 folder structure. If your URL is https://example.com/media/mediafile.mp4, then there needs to be a folder structure like that. bucketname/media/mediafile.mp4.

Let me know if the above works. If not, we can dive a bit deeper.

kentrad · Answer

You can place the media files in the same bucket as the web site, however, you add an additional Behavior to the CloudFront Distribution with a specific PathPattern like "*.mpeg" and use the CachingDisabled policy so that the videos are treated as dynamic content.

Harshavardhan_G · Answer

Hello Gabriel,

Full disclosure, I am not an S3 support engineer. I found the query interesting.

I am not sure if you have tried CORS yet. If not, you can find more details here:\
[+] https://docs.aws.amazon.com/AmazonS3/latest/userguide/ManageCorsUsing.html

Maybe try allowing your domain and the cloudfront's domain to access the secondary bucket and use the URL to the objects in your code?
```
"AllowedOrigins": [
            "http://www.example1.com"
```

Please let me know if it works. If not, maybe I will try digging through a few documents.

Take care and stay safe !

Cloudfront X S3 website + secondary bucket for specific media

S3 Bucket

CloudFront

Relevant content