By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

EC2 of private subnet to S3 bucket flow log

0

Topology: I have EC2 instances hosted in a private subnet. This EC2 instance communicates to S3 bucket through NAT gateway.

Requirement: Analyse "which EC2 communicates with which S3 bucket (bucket name)".

Failed experiments to get EC2 to S3 bucket log:

  1. Using VPC flow logs of subnet: It has S3 bucket IP as destination IP. But as S3 bucket IP is ephemeral. So, it cannot be co-related to S3 bucket name.
  2. Using NAT flow logs: It has EC2 IP, S3 IP and NAT IP. Again as S3 IP is ephemeral, it cannot be co-related to S3 bucket name
  3. Using Cloudtrail logs of S3 bucket: It has NAT IP as Source IP instead of EC2 IP.

Question: Is there any way/logs by which I can get which EC2 is talking to which S3 bucket information directly or by co-relating 2 or more logs? Is there any setting that needs to be done to get this information.

Topology

Topics
Management & GovernanceStorageNetworking & Content Delivery
Tags
AWS CloudTrailS3 SelectVPC Flow Logs
Language
English
rePost-User-1968636
asked a year ago296 views
1 Answer
0
Accepted Answer

Flow logs are not the appropriate tool to use here because (as you've discovered) the bucket name isn't related to the IP address. Instead, it is part of the higher-level protocol (it's transmitted as part of the HTTPS request to the S3 API) - so even if you were doing deep-packet inspection it would still be encrypted which makes it difficult (again) to determine.

To get the information you require, you might try the following:

  • Create a S3 Gateway Endpoint - this has no extra charge (in fact, it may reduce your NAT Gateway charges) and is transparent to your application(s).
  • Enable S3 access logging - note that you will pay for storage of the logs so I'd recommend deleting old logs to save costs.
  • The access log format has the source IP address of the requester in it.

Followup question: What do you need this information for?

profile pictureAWS
EXPERT
Brettski-AWS
answered a year ago
  • rePost-User-1968636
    a year ago

    Thanks for the answer! We need this information to monitor communication between EC2 and S3 for some analytics purpose.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content