By using AWS re:Post, you agree to the Terms of Use
/Problem uploading media to AWS S3 with Django Storages / Boto3 (form a website on Lambda)/

Problem uploading media to AWS S3 with Django Storages / Boto3 (form a website on Lambda)

0

Hi all! I have a Django website which is deployed on AWS Lambda. All the static/media is stored in the S3 bucket. I managed to serve static from S3 and it works fine, however, when trying to upload media through admin (I was trying to add an article with a pic attached to it), I get a message "Endpoint request timed out".

Here is my AWS and storage configuration:

ukraine101.aws.utils.py

from storages.backends.s3boto3 import S3Boto3Storage

StaticRootS3BotoStorage = lambda: S3Boto3Storage(location='static')
MediaRootS3BotoStorage  = lambda: S3Boto3Storage(location='media')

settings.py

STATICFILES_DIRS = [BASE_DIR / "static"]
STATIC_URL = 'https://<my-bucket-name>.s3.amazonaws.com/'
MEDIA_URL = 'https://<my-bucket-name>.s3.amazonaws.com/media/'
MEDIA_ROOT = MEDIA_URL

DEFAULT_FILE_STORAGE = 'ukraine101.aws.utils.MediaRootS3BotoStorage'
STATICFILES_STORAGE = 'ukraine101.aws.utils.StaticRootS3BotoStorage'

AWS_STORAGE_BUCKET_NAME = '<my-bucket-name>'
AWS_S3_REGION_NAME = 'us-east-1'  
AWS_ACCESS_KEY_ID = '<my-key-i-dont-show>'
AWS_SECRET_ACCESS_KEY = '<my-secret-key-i-dont-show>'
AWS_S3_SIGNATURE_VERSION = 's3v4'
AWS_S3_FILE_OVERWRITE = False
AWS_DEFAULT_ACL = None
AWS_S3_VERIFY = True

AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
STATICFILES_LOCATION = 'static'

My Article model:

class Article(models.Model):

    title = models.CharField(max_length=250, )
    summary = models.TextField(blank=False, null=False, )
    image = models.ImageField(blank=False, null=False, upload_to='articles/', )
    text = RichTextField(blank=False, null=False, )
    category = models.ForeignKey(Category, null=True, blank=True, default='', on_delete=models.SET_DEFAULT)
    featured = models.BooleanField(default=False)
    date_created = models.DateField(auto_now_add=True)
    slug = AutoSlugField(populate_from='title')
    related_book = models.ForeignKey(Book, null=True, blank=True, default='', on_delete=models.SET_DEFAULT)

    def get_absolute_url(self):
        return reverse("articles:article-detail", kwargs={"slug": self.slug})

    def get_comments(self):
        return Comment.objects.filter(article=self.id)


    author = models.ForeignKey(User, null=True, blank=True, default='', on_delete=models.SET_DEFAULT)

AWS bucket policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PublicRead",
            "Effect": "Allow",
            "Principal": "*",
            "Action": [
                "s3:PutObject",
                "s3:PutObjectAcl",
                "s3:GetObject",
                "s3:GetObjectVersion",
                "s3:GetObjectAcl"
            ],
            "Resource": "arn:aws:s3:::<my-bucket-name>/*"
        }
    ]
}

CORS:

[
    {
        "AllowedHeaders": [
            "*"
        ],
        "AllowedMethods": [
            "GET",
            "POST",
            "PUT",
            "HEAD"
        ],
        "AllowedOrigins": [
            "*"
        ],
        "ExposeHeaders": [],
        "MaxAgeSeconds": 3000
    }
]

**User permissions policies (there are two attached): ** Policy 1:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListAllMyBuckets"
            ],
            "Resource": "arn:aws:s3:::*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation",
                "s3:ListBucketMultipartUploads",
                "s3:ListBucketVersions"
            ],
            "Resource": "arn:aws:s3:::<my-bucket-name>"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:*Object*",
                "s3:ListMultipartUploadParts",
                "s3:AbortMultipartUpload"
            ],
            "Resource": "arn:aws:s3:::<my-bucket-name>/*"
        }
    ]
}

Policy 2:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:*",
                "s3-object-lambda:*"
            ],
            "Resource": [
                "arn:aws:s3:::<my-bucket-name>",
                "arn:aws:s3:::<my-bucket-name>/*"
            ]
        }
    ]
}

Please, if someone knows what can be wrong and why this timeout is happening, help me.

1 Answers
0

If the media you are uploading is greater than 5 GB, the SDK will use the multi-part APIs and you'll need permissions to use them: CreateMultipartUpload, Upload Part, and CompleteMultipartUpload. You may also need permissions to AbortMultipartUpload and ListMultipartUploads to handle error conditions.

answered 2 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions