S3 permissions for MediaTailor

I am trying to setup permissions for an S3 bucket so that a particular "folder"[1] can be accessed by the MediaTailor service.

I need to allow MediaTailor to be able to read all objects in the "videos" folder.
I am using root account for my experiments.
The objects inside "videos" have only 1 permission set - full access to owner (root account).
S3 bucket has "block public access settings" turned on.
I prefer not having per-object permissions since I might have thousands of objects (to be scalable to millions).

Based on my understanding, if I add the following to S3 bucket policy, it should be enough to allow MediaTailor to read objects inside "videos" folder.

{
    "Version": "2008-10-17",
    "Id": "PolicyForMediaTailor",
    "Statement": [
        {
            "Sid": "1",
            "Effect": "Allow",
            "Principal": {
                "Service": "mediatailor.amazonaws.com"
            },
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::mybucket/videos/*"
        }
    ]
}

However, when I try to access an object inside videos folder via MediaTailor, it results in http 403 (forbidden) error.

Does anyone know what I am doing wrong?
Thanks!

[1] I am aware that there are no "folders" in S3.