My AWS Batch job service uses a lot of space to S3, while my code doesn't work with S3?

0

I have 5 Batch jobs running on AWS Batch with Fargate, when it was running I noticed the capacity to S3 spiked through the NAT Gateway. I queried VPC Logs using Athena and found that the destination IP is of S3 None of my code uses S3, when I turn them off the capacity going to S3 is completely reduced. I don't understand why my Batch job service uses S3 while my code doesn't. Is there any way to investigate to know exactly where the capacity is coming from? (except https://aws.amazon.com/premiumsupport/knowledge-center/vpc-find-traffic-sources-nat-gateway/?nc1=h_ls , I read it). I understand it is possible to use S3 VPC Endpoint to handle throughput through the NAT Gateway, but I want to find the root cause.

1개 답변
1
수락된 답변

Your container image (which I assume is in ECR) is stored in s3 which is the likely cause of the s3 traffic spike. An s3 gateway endpoint can be setup to optimise the download - see here

AWS
전문가
Peter_G
답변함 2년 전
profile pictureAWS
전문가
검토됨 2년 전
  • Ok many thanks, exactly I am using ECR but I remember I saw somewhere on the internet wrote that ECR is using S3 to store images, where can I find evidence of this (eg an AWS blog, tutorial link, ...). I need to prove something...

  • From the link I posted above - "When your containers download images from Amazon ECR, they must access Amazon ECR to get the image manifest and then Amazon S3 to download the actual image layers"

  • Thank you so much

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인