Problem uploading media to AWS S3 with Django Storages / Boto3 (form a website on Lambda)
Hi all! I have a Django website which is deployed on AWS Lambda. All the static/media is stored in the S3 bucket. I managed to serve static from S3 and it works fine, however, when trying to upload media through admin (I was trying to add an article with a pic attached to it), I get a message "Endpoint request timed out".
Here is my AWS and storage configuration:
ukraine101.aws.utils.py
from storages.backends.s3boto3 import S3Boto3Storage
StaticRootS3BotoStorage = lambda: S3Boto3Storage(location='static')
MediaRootS3BotoStorage = lambda: S3Boto3Storage(location='media')
settings.py
STATICFILES_DIRS = [BASE_DIR / "static"]
STATIC_URL = 'https://<my-bucket-name>.s3.amazonaws.com/'
MEDIA_URL = 'https://<my-bucket-name>.s3.amazonaws.com/media/'
MEDIA_ROOT = MEDIA_URL
DEFAULT_FILE_STORAGE = 'ukraine101.aws.utils.MediaRootS3BotoStorage'
STATICFILES_STORAGE = 'ukraine101.aws.utils.StaticRootS3BotoStorage'
AWS_STORAGE_BUCKET_NAME = '<my-bucket-name>'
AWS_S3_REGION_NAME = 'us-east-1'
AWS_ACCESS_KEY_ID = '<my-key-i-dont-show>'
AWS_SECRET_ACCESS_KEY = '<my-secret-key-i-dont-show>'
AWS_S3_SIGNATURE_VERSION = 's3v4'
AWS_S3_FILE_OVERWRITE = False
AWS_DEFAULT_ACL = None
AWS_S3_VERIFY = True
AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
STATICFILES_LOCATION = 'static'
My Article model:
class Article(models.Model):
title = models.CharField(max_length=250, )
summary = models.TextField(blank=False, null=False, )
image = models.ImageField(blank=False, null=False, upload_to='articles/', )
text = RichTextField(blank=False, null=False, )
category = models.ForeignKey(Category, null=True, blank=True, default='', on_delete=models.SET_DEFAULT)
featured = models.BooleanField(default=False)
date_created = models.DateField(auto_now_add=True)
slug = AutoSlugField(populate_from='title')
related_book = models.ForeignKey(Book, null=True, blank=True, default='', on_delete=models.SET_DEFAULT)
def get_absolute_url(self):
return reverse("articles:article-detail", kwargs={"slug": self.slug})
def get_comments(self):
return Comment.objects.filter(article=self.id)
author = models.ForeignKey(User, null=True, blank=True, default='', on_delete=models.SET_DEFAULT)
AWS bucket policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "PublicRead",
"Effect": "Allow",
"Principal": "*",
"Action": [
"s3:PutObject",
"s3:PutObjectAcl",
"s3:GetObject",
"s3:GetObjectVersion",
"s3:GetObjectAcl"
],
"Resource": "arn:aws:s3:::<my-bucket-name>/*"
}
]
}
CORS:
[
{
"AllowedHeaders": [
"*"
],
"AllowedMethods": [
"GET",
"POST",
"PUT",
"HEAD"
],
"AllowedOrigins": [
"*"
],
"ExposeHeaders": [],
"MaxAgeSeconds": 3000
}
]
**User permissions policies (there are two attached): ** Policy 1:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListAllMyBuckets"
],
"Resource": "arn:aws:s3:::*"
},
{
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetBucketLocation",
"s3:ListBucketMultipartUploads",
"s3:ListBucketVersions"
],
"Resource": "arn:aws:s3:::<my-bucket-name>"
},
{
"Effect": "Allow",
"Action": [
"s3:*Object*",
"s3:ListMultipartUploadParts",
"s3:AbortMultipartUpload"
],
"Resource": "arn:aws:s3:::<my-bucket-name>/*"
}
]
}
Policy 2:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:*",
"s3-object-lambda:*"
],
"Resource": [
"arn:aws:s3:::<my-bucket-name>",
"arn:aws:s3:::<my-bucket-name>/*"
]
}
]
}
Please, if someone knows what can be wrong and why this timeout is happening, help me.
If the media you are uploading is greater than 5 GB, the SDK will use the multi-part APIs and you'll need permissions to use them: CreateMultipartUpload, Upload Part, and CompleteMultipartUpload. You may also need permissions to AbortMultipartUpload and ListMultipartUploads to handle error conditions.
Relevant questions
S3 object and lambda function in step machine with the asynchronous express workflows
Accepted Answerasked 5 months agouse django developing web site in aws ec2
asked a month agoOAI or not OAI for serving a static website in S3 using CloudFront
asked 3 months agoUploading a Dataframe to AWS S3 Bucket from SageMaker
Accepted Answerasked 3 years agoASP.NET Web API on AWS Lambda CORS
asked 4 months agoServerless web form
asked 6 months agoHow to delete old files deployed in S3 with CodePipeline
asked 2 months agoIs it possible to map an S3 static website to a public static IP address?
Accepted Answerasked 2 years agoProblem uploading media to AWS S3 with Django Storages / Boto3 (form a website on Lambda)
asked 2 months agoGenerate presigned url for S3 Object Lambda
Accepted Answerasked a year ago