View raw data capture

Hi Mystic,

You have a few options here. Do you need to capture layer 7 traffic (like HTTP headers/responses), or full packet captures?

If you are managing your BigCommerce hosting yourself on EC2, you can set up packet capture on your hosted instance.

If you want to capture HTTP, then a package like mitmproxy or squid can act as a proxy and log all traffic. Note, you will need to do this after SSL termination, so that you can read HTTP headers in clear text. This will work fine if you have an Application Load Balancer terminating SSL, because traffic inside your VPC will be decrypted and readable.

You can use a tool like tcpdump on Linux or WireShark on Windows to capture a packet dump. You can then analyse it using the WireShark GUI on the platform of your choice.

You can see an example architecture demonstrating that model here: https://aws.amazon.com/blogs/security/how-to-set-up-an-outbound-vpc-proxy-with-domain-whitelisting-and-content-filtering/

If you are not managing BigCommerce directly or would prefer to use AWS tooling, there are two VPC features that may be of use.

VPC Flow Logs

You can use VPC Flow Logs to capture the packet headers. Flow Logs are for capturing information about how traffic flows - fundamentally, its source, destination, and status - rather than capturing all of the traffic and the data within in. You can include a lot of additional data in your logs but not the data content of packets.

Flow logs can help you with a number of tasks, such as:

• Diagnosing overly restrictive security group rules
• Monitoring the traffic that is reaching your instance
• Determining the direction of the traffic to and from the network interfaces

More information about how to set up Flow Logs, the data you can capture, and tooling for storing and analysing the logs is available here: https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs.html

VPC Traffic Mirroring

VPC Traffic Mirroring allows to capture all network traffic from a source Elastic Network Interface (ENI). You specify a source - for example, the ENI attached to one of your BigCommerce application servers, then you mirror that traffic to a destination. A destination can be either:

• Another ENI
• A Network Load Balancer (NLB)
• A Gateway Load Balancer endpoint

In all three cases, the target will simply receive the traffic. You will then need to decide how to log and inspect it.

For security use cases, we have documented Zeek and Suricata as options: https://docs.aws.amazon.com/vpc/latest/mirroring/tm-example-open-source.html

For raw capture, then running WireShark on an EC2 instance might be a good option.

You can read more about traffic mirroring in our blog announcing the feature: https://aws.amazon.com/blogs/aws/new-vpc-traffic-mirroring/