Problem generating signed URL with CloudFront

0

I have the following configuration in cloudfront

Origins

Origin nameOrigin domainOrigin pathOrigin typeOrigin Shield regionOrigin access
bucket-user-files.s3.us-east-2.amazonaws.combucket-user-files.s3.us-east-2.amazonaws.comS3us-east-2E29MY3
bucket-events.s3.us-east-2.amazonaws.combucket-events.s3.us-east-2.amazonaws.comS3us-east-2E2AYU

in both of them I have activated the Origin Access option: Origin Access Control Settings (recommended)

Behaviors

PrecedencePath patternOrigin or origin groupViewer protocol policyCache policy nameOrigin request policy nameResponse headers policy name
0events/*bucket-events.s3.us-east-2.amazonaws.comRedirect HTTP to HTTPSTW_CF_S3_UserFIlesTW_CF_S3_UserFIles_Origin-
1user-files/*bucket-user-files.s3.us-east-2.amazonaws.comRedirect HTTP to HTTPSTW_CF_S3_UserFIlesTW_CF_S3_UserFIles_Origin-
2Default (*)bucket-user-files.s3.us-east-2.amazonaws.comRedirect HTTP to HTTPSTW_CF_S3_UserFIlesTW_CF_S3_UserFIles_Origin-
  • in both I have Viewer option active: HTTP methods allowed: GET, HEAD, OPTIONS -> Cache HTTP mwthods: OPTIONS
  • in both in Restrict viewer access, I have it set to yes -> Trusted key groups -> post-files (key group)
  • in both I use the same key group in both in Cache key and origin requests I have selected the option "Cache policy and origin request policy".

S3 buckets: in both buckets I have the same policies only differing in the resource, for their respective bucket.

bucket-user-files

{
    "Version": "2008-10-17",
    "Id": "PolicyForCloudFrontPrivateContent",
    "Statement": [
        {
            "Sid": "AllowCloudFrontServicePrincipal",
            "Effect": "Allow",
            "Principal": {
                "Service": "cloudfront.amazonaws.com"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::bucket-user-files/*",
            "Condition": {
                "StringEquals": {
                    "AWS:SourceArn": "arn:aws:cloudfront::084135377906:distribution/EV633TPE1OT1W"
                }
            }
        }
    ]
}

bucket-events

{
    "Version": "2008-10-17",
    "Id": "PolicyForCloudFrontPrivateContent",
    "Statement": [
        {
            "Sid": "AllowCloudFrontServicePrincipal",
            "Effect": "Allow",
            "Principal": {
                "Service": "cloudfront.amazonaws.com"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::bucket-events/*",
            "Condition": {
                "StringEquals": {
                    "AWS:SourceArn": "arn:aws:cloudfront::084135377906:distribution/EV633TPE1OT1W"
                }
            }
        }
    ]
}

in both buckets I have the following configuration Block public access (bucket configuration)

Block all public access: Disabled

Not selected: Block public access to buckets and objects granted through new access control lists (ACL)

Not selected : Block public access to buckets and objects granted through any Access Control List (ACL)

selected: Block public access to buckets and objects granted through new bucket policies and public access points

selected: Block public and inter-account access to buckets and objects granted through any public bucket and access point policy.

Access Control List (ACL)

RecipientObjectsBucket ACLs
Bucket owner (your AWS account)List, Read, WriteRead, Write
Everyone (public access)--
Group of authenticated users (anyone with an AWS account)--
Group S3 log forwarding--

Cross-Origin Resource Sharing (CORS)

[
    {
        "AllowedHeaders": [
            "*"
        ],
        "AllowedMethods": [
            "GET"
        ],
        "AllowedOrigins": [
            "*"
        ],
        "ExposeHeaders": []
    }
]

now, in my code I have the following

<?php

namespace App\Service\AWS;

use Aws\CloudFront\CloudFrontClient;
use Aws\Exception\AwsException;
use DateInterval;
use DateTime;
use Symfony\Contracts\Cache\ItemInterface;
use Symfony\Contracts\Cache\TagAwareCacheInterface;

class CloudFront
{
    const EXPIRATION_TIME_CLOUDFRONT_URL = 'PT8H';
    const CACHE_PUBLIC_URL_TIME = 'P2M';


    private CloudFrontClient $cloudFrontClient;
    private TagAwareCacheInterface $cache;
    private string $keyPath;
    private string $keyId;
    private string $distributionUrl;
    private string $origin;

    public function __construct(
        CloudFrontClient $cloudFrontClient,
        TagAwareCacheInterface $cache,
        string $PRIVATE_KEY_ID,
        string $PRIVATE_KEY_PATH,
        string $DISTRIBUTION_URL,
    ) {
        $this->cloudFrontClient = $cloudFrontClient;
        $this->cache = $cache;
        $this->keyId = $PRIVATE_KEY_ID;
        $this->keyPath = $PRIVATE_KEY_PATH;
        $this->distributionUrl = $DISTRIBUTION_URL;
    }

    public function getSignedUrl(string $origin, string $key, string $ip): string {
        $expires = new DateTime();
        $expires->add(new DateInterval(self::EXPIRATION_TIME_CLOUDFRONT_URL));

        if ($ip == "::1" || $ip == "127.0.0.1") { // for test in local
            $ip = file_get_contents('https://ipecho.net/plain');
        }

        $resource = $this->distributionUrl.$origin."/".$key;
        $customPolicy = <<<POLICY
        {
            "Statement": [
                {
                    "Resource": "{$resource}",
                    "IpAddress": {"AWS:SourceIp": "{$ip}/32"},
                    "Condition": {
                        "DateLessThan": {"AWS:EpochTime": {$expires->getTimestamp()}}
                    }
                }
            ]
        }
        POLICY;

        try {
            return $this->cloudFrontClient->getSignedUrl([
                'url' => $resource,
                'policy' => $customPolicy,
                'private_key' => $this->keyPath,
                'key_pair_id' => $this->keyId
            ]);

        } catch (AwsException $e) {
            return 'Error: ' . $e->getAwsErrorMessage();
        }
    }

    /**
     * @throws \Psr\Cache\InvalidArgumentException
     */
    public function getPublicUrl(string $origin, string $key)
    {
        $itemName = htmlspecialchars($key);
        $itemName = str_replace('/','',$itemName);

        return $this->cache->get("public_".$itemName."_file",
            function (ItemInterface $item) use ($origin, $key) {
                $cachedTime = new DateInterval(self::CACHE_PUBLIC_URL_TIME);
                $item->expiresAfter($cachedTime);
                $item->tag(['public-url-files']);

                $expires = new DateTime();
                $expires->add($cachedTime);

                try {
                    return $this->cloudFrontClient->getSignedUrl([
                        'url' => $this->distributionUrl.$origin."/".$key,
                        'expires' => $expires->getTimestamp(),
                        'private_key' => $this->keyPath,
                        'key_pair_id' => $this->keyId
                    ]);

                } catch (AwsException $e) {
                    return 'Error: ' . $e->getAwsErrorMessage();
                }
            }
        );
    }
}

when I call getSignedUrl() when executing the return $this->cloudFrontClient->getSignedUrl I pass the parameters:

url => https://d1andaiyzi47n1.cloudfront.net/user-files/e230a742-e5d5-4c51-ac76-32dc1ceadb91/post/838e9873-0c65-4525-a563-87a0cfb8b283/64f4e586a3cae4.40728513.png
policy => {
    "Statement": [
        {
            "Resource": "https://d1andaiyzi46n1.cloudfront.net/user-files/e230a742-e5d5-4c51-ac76-32dc1ceadb91/post/838e9873-0c65-4525-a563-87a0cfb8b283/64f4e586a3cae4.40728513.png",
            "IpAddress": {"AWS:SourceIp": "***."***.."***.."***./32"},
            "Condition": {
                "DateLessThan": {"AWS:EpochTime": 1693799943}
            }
        }
    ]
}

at the end it does return a url that appears to be signed correctly e.g.

https://d1andaiyzi46n1.cloudfront.net/user-files/e230a742-e5d5-4c51-ac76-32dc1ceadb91/post/838e9873-0c65-4525-a563-87a0cfb8b283/64f4e586a3cae4.40728513.png?Policy=eyJT...mh-79n8SA__&Key-Pair-Id=K2Z8L65Y2UGBCT

I see this url in the time limit that I set as well as from the ip that I set. however when I open that url signed in the browser I get the following message

This XML file does not appear to have any style information associated with it. The document tree is shown below.
<Error>
<Code>AccessDenied</Code>
<Message>Access Denied</Message>
<RequestId>5EAYW06QE5218T9T</RequestId>
<HostId>smMq546FKNMqAUNepRJu/bU1FNY3oL9OQtR28JE0UcqLQir5uYVjyMDHsstnpLrU0tHsTPU/N48=</HostId>
</Error>

I am also doing the manual test from the aws cli and the same thing happens, it says access denied.

here is the aws command

aws cloudfront sign --url 'https://d1andaiyzi46n1.cloudfront.net/user-files/e230a742-e5d5-4c51-ac76-32dc1ceadb91/post/838e9873-0c65-4525-a563-87a0cfb8b283/64f4e586a3cae4.40728513.png' --key-pair-id 'K2Z8L65Y2UCBGT' --private-key file://C:/Users/.../secrets/cloudfront-post-files-key1_private.pem --date-less-than '2024-09-04T12:00:00Z'

The logs that cloudfront generates do not give much information only that it seems to be some configuration or permissions issue, but it does not give any indication of what it could be. This is supported by the fact that as soon as the request arrives to cloudfront immediately launches the log with the error, without having a reasonable waiting time to give rise to another diagnosis.

It should be noted that I already had a configuration with only one origin working correctly, however when I tried to add another origin from a different s3 bucket is when neither of the two origins worked.

note: all ids, links and service names are not the real ones. they were modified for the purpose of making the details of the problem as clear as possible.

2 Answers
0
Accepted Answer

While working with AWS CloudFront having multiple origins, I encountered an "Access Denied" issue. After some investigation, I found the root cause and a solution.

Problem:

I mistakenly believed that by defining a unique path pattern (Path Pattern) for each behavior according to the bucket, and then constructing the CloudFront request URL with that path pattern, CloudFront would automatically route to the appropriate origin. Unfortunately, it doesn't work that way.

Even if you define a distinct path pattern for each origin, when the constructed CloudFront URL is accessed, it attempts to fetch the file from the S3 bucket, including the path pattern itself. For example, I was constructing URLs that looked like this: https://cloudfront-distribution/user-files/[file-key] I assumed that CloudFront would use the user-files/* path pattern to route to the correct S3 bucket origin. However, in reality, it was looking for a folder named user-files/ within the S3 bucket. My S3 bucket didn't have a folder named user-files/. Instead, it contained several folders named by user UUIDs.

Solution:

Understanding the issue led me to the conclusion that the path patterns aren't to be used as mere routing indicators in CloudFront. Instead, they should represent the actual structure within the S3 bucket.

Given that I had user-specific UUID folders within the S3 bucket, it wasn't feasible to define a path pattern for each of them. The most straightforward solution was to create a separate CloudFront distribution for each S3 bucket, avoiding the need for additional path patterns.

In conclusion, if you're facing a similar "Access Denied" issue and you're using CloudFront with multiple origins and path patterns, double-check how you're constructing your URLs. Ensure that the path patterns align with the actual structure of your S3 bucket, or consider using separate distributions for clarity and simplicity.

I hope this helps anyone facing a similar issue!

answered 7 months ago
0

Hi there,

The first what I can see is that you have:

Block all public access: Disabled

Not selected: Block public access to buckets and objects granted through new access control lists (ACL)

Not selected : Block public access to buckets and objects granted through any Access Control List (ACL)

selected: Block public access to buckets and objects granted through new bucket policies and public access points

selected: Block public and inter-account access to buckets and objects granted through any public bucket and access point policy.

but at the same time you are using policies to grant access to the S3 bucket/objects. I believe your settings should be reversed (have the first two SELECTED and the other NOT SELECTED)

profile picture
EXPERT
answered 7 months ago
  • Thanks for your comment. I am trying to configure 'Private Content' in CloudFront, where the goal is to make the S3 bucket only accessible via signed CloudFront URLs and block direct access to S3. The current configuration ensures that public access cannot be granted through bucket policies or public access points. Allowing ACLs (as you suggest) could open up possibilities for direct public access, which is not desired in this scenario.

  • I'm trying to pay attention to the fact that your logic is reversed. When it's "Not selected" - meaning this type of access is NOT blocked. Your policy you assigned to grant access only for CF just not working because you blocked using policies but intention was to block ACL (which are not blocked)

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions