My AWS Batch job service uses a lot of space to S3, while my code doesn't work with S3?

0

I have 5 Batch jobs running on AWS Batch with Fargate, when it was running I noticed the capacity to S3 spiked through the NAT Gateway. I queried VPC Logs using Athena and found that the destination IP is of S3 None of my code uses S3, when I turn them off the capacity going to S3 is completely reduced. I don't understand why my Batch job service uses S3 while my code doesn't. Is there any way to investigate to know exactly where the capacity is coming from? (except https://aws.amazon.com/premiumsupport/knowledge-center/vpc-find-traffic-sources-nat-gateway/?nc1=h_ls , I read it). I understand it is possible to use S3 VPC Endpoint to handle throughput through the NAT Gateway, but I want to find the root cause.

1 Antwort
1
Akzeptierte Antwort

Your container image (which I assume is in ECR) is stored in s3 which is the likely cause of the s3 traffic spike. An s3 gateway endpoint can be setup to optimise the download - see here

AWS
EXPERTE
Peter_G
beantwortet vor 2 Jahren
profile pictureAWS
EXPERTE
überprüft vor 2 Jahren
  • Ok many thanks, exactly I am using ECR but I remember I saw somewhere on the internet wrote that ECR is using S3 to store images, where can I find evidence of this (eg an AWS blog, tutorial link, ...). I need to prove something...

  • From the link I posted above - "When your containers download images from Amazon ECR, they must access Amazon ECR to get the image manifest and then Amazon S3 to download the actual image layers"

  • Thank you so much

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen

Relevanter Inhalt