Unable to See HTTP Logs in AWS Network Firewall Cloudwatch -- Only UDP Logs

0

For context I am following the instructions posted here:

https://repost.aws/knowledge-center/network-firewall-set-up-with-nat-gateway

📌 Questions for AWS Support
1️⃣ Why is AWS Network Firewall not logging HTTP/TLS (TCP 80/443) traffic?
2️⃣ Is AWS Network Firewall dropping HTTP/TLS traffic before reaching the NAT Gateway?
3️⃣ How can I verify if the NAT Gateway is forwarding HTTP/TLS traffic correctly?
Here is my setup: I have AWS Network Firewall deployed to inspect outbound internet traffic from private subnets before forwarding it to a NAT Gateway. The expected flow is:
Private Subnets → AWS Network Firewall VPCE Firewall Subnets → NAT Gateway NAT Gateway → Internet Gateway.

✅ What is Working:. DNS traffic (UDP 53) is logged in AWS Network Firewall. All private subnets correctly route 0.0.0.0/0 to AWS Network Firewall VPCEs. Firewall subnets route 0.0.0.0/0 to the NAT Gateway. AWS Network Firewall logging is enabled for Flow & Alert logs. The Network ACLs for the subnets allow traffic to the corresponding places that I expect.

❌ Issue:.
No HTTP (TCP 80) or HTTPS (TCP 443) logs appear in AWS Network Firewall.
Traffic might be dropping before reaching the NAT Gateway.

📌 Network ACL Configuration.
🔹 Network ACL for NAT Gateway Subnet.
Rule Number Direction Protocol Port Range Source/Destination Action. 101 Egress TCP 80 0.0.0.0/0 ALLOW. 102 Egress TCP 443 0.0.0.0/0 ALLOW. 103 Ingress TCP 1024-65535 0.0.0.0/0 ALLOW.

✅ Expected Outcome:. NAT Gateway should allow outbound HTTP/TLS requests and receive inbound responses from the internet.

🔹 Network ACL for Private Subnets. Rule Number Direction Protocol Port Range Source/Destination Action. 101 Egress All All 10.12.48.0/20 (Firewall VPCE) ALLOW. 102 Egress All All 10.12.64.0/20 (Firewall VPCE) ALLOW. 103 Egress All All 10.12.128.0/20 (Firewall VPCE) ALLOW. 104 Egress All All 10.12.144.0/20 (Firewall VPCE) ALLOW. 32767 Egress All All 0.0.0.0/0 DENY.

✅ Expected Outcome:. Only traffic through AWS Network Firewall should be allowed.
Direct access to NAT Gateway is blocked.

🔹 Network ACL for Firewall Subnets. Rule Number Direction Protocol Port Range Source/Destination Action. 201 Egress All All 10.12.48.0/20 (Firewall VPCE) ALLOW. 202 Egress All All 10.12.64.0/20 (Firewall VPCE) ALLOW. 203 Egress All All 10.12.128.0/20 (Firewall VPCE) ALLOW. 204 Egress All All 10.12.144.0/20 (Firewall VPCE) ALLOW. ✅ Expected Outcome:. Firewall subnets should send traffic to NAT Gateway after inspection.
Traffic must be logged before being forwarded to NAT Gateway.


3 Answers
1
Accepted Answer

It could be a number of things.

  1. Without seeing your ANFW policy, rule groups and routing - it's hard to say why these things are occurring.
  2. I'm assuming, based on what your stated that UDP 53 traffic is logging correctly, and reaching its destination? If ANFW is logging UDP 53 - are you not using the AWS Route 53 Resolver? If you are, that UDP resolution traffic takes an entirely different path (service backplane) than non-DNS 53 traffic and shouldn't be logged in the ANFW.
  3. What is your default deny rule stating? Are you using Drop-Established? Drop-All? Or nothing? If TCP traffic isn't making it to the ANFW, than it could be a default deny rule.
  4. Do you have Alert Rules, in front of Pass rules? Alert rules are non-terminating rules, and they alert. Pass rules are terminating and DO NOT LOG.
  5. Do you have routing configured appropriately, as in the outbound and return paths are routed the same - focus on the return routes for the NAT GW public-subnet (please see the URL for verifying this): https://aws.amazon.com/blogs/networking-and-content-delivery/deployment-models-for-aws-network-firewall-with-vpc-routing-enhancements/
  6. Check your IP address by performing a curl to https://checkip.amazonaws.com - if it's not reachable, possibly one of the above is true. If it is reachable, and it's not the IP address of your EC2, it's most likely the IP of the NAT GW.
  7. I'm not sure if you're using Suricata Custom Rules (manually coded) or standard Stateful Rules (GUI). So here is a great webpage to help sort that out (if you're using Suricata Rules): https://docs.aws.amazon.com/network-firewall/latest/developerguide/suricata-examples.html

Two of things most often seen are Pass rule in front of the alert rule --> no logging and Suricata rules being written with contradicting depth inspection, as an example:

#pass tls any any -> any any (tls.sni; content:"aws.amazon.com"; startswith; nocase; endswith; msg:"Permit HTTPS access to aws.amazon.com"; sid:1000002; rev:1;) #drop tcp any any -> any any (msg:"Deny all other TCP traffic"; sid: 1000003; rev:1;)

As written, the first rule will never match because the TCP protocol is the first protocol that will appear in the initial flow, starting with the TCP handshake. Therefore, when the flow starts, while the first rule will be evaluated, there will be no TLS protocol to match, so the drop rule will match instead and drop all of the traffic to aws.amazon.com. To avoid this, you should write rules that only evaluate after a session has been established using the flow keyword (TLS could be replaced with HTTP or HTTPS).

CORRECTLY WRITTEN: #pass tls any any -> any any (tls.sni; content:"aws.amazon.com"; startswith; nocase; endswith; msg:"Permit HTTPS access to aws.amazon.com"; sid:1000002; rev:1;) #drop tcp any any -> any any (flow:established,to_server; msg:"Deny all other TCP traffic"; sid: 1000003; rev:1;)

Using “flow:to_server” in the rules will make the rules operate at the same level and evaluate the traffic at the same time so that the pass rule has a chance to inspect the traffic before the reject rule blocks the traffic.

I hope this helps and you get your issue resolved!

Oh and here's our ANFW best practices github doc, regularly updated: https://aws.github.io/aws-security-services-best-practices/guides/network-firewall/

AWS
answered 2 months ago
profile picture
EXPERT
reviewed a month ago
  • Thanks for answering! and for all the resources So I changed it to just have these rules to test alerting: (I am doing stateless -> Forward to stateful rule groups And Actions for fragmented packets: Pass) And then strict order for stateful: alert tcp any any -> any any (msg:"ALERT: TCP Traffic Logged"; sid:9999991;) alert udp any any -> any any (msg:"ALERT: UDP Traffic Logged"; sid:9999992;) alert icmp any any -> any any (msg:"ALERT: ICMP Traffic Logged"; sid:9999993;) alert dns any any -> any any (msg:"ALERT: DNS Traffic Logged"; sid:9999994;) alert http any any -> any any (msg:"ALERT: HTTP Traffic Logged"; sid:9999995;) alert http any any -> any any (msg:"ALERT: HTTP Traffic Detected"; flow:to_server; sid:9999997;) alert tls any any -> any any (msg:"ALERT: TLS Traffic Logged"; sid:9999996;) alert tls $HOME_NET any -> any any (tls.sni; content:"www.example.com"; startswith; nocase; endswith; flow:to_server; msg:"TLS SNI alert example"; sid:202501052;) reject tls $HOME_NET any -> any any (tls.sni; content:"www.example.com"; startswith; nocase; endswith; flow:to_server; msg:"reject example.com"; sid:202501053;)

    I can see DNS,TCP,UDP logs now but never HTTP? When I do curl https://checkip.amazonaws.com in an ec2 I do get the NAT gateway IP. with the above i would expect that ill see example.com be blocked, but I am still able to curl it from inside my VPC I want to make sure that I am able to HTTP report generations so I can get a list of domains

  • Thank you for the response. Let me give this some thought - I'll respond with some additional comments later as well:

    Initially response, "assuming" routing is appropriately setup with provided alert logs (those are coming from the ANFW-Alert log in CW logs?)

    What are your default rules in the stateful policy? (true allow list for domains would need Drop-Established and Alert-Established.

    If you're alerting on www.example.com, but curling example.com (sub-domain vs root domain). Potential issue there - you're inspecting for TLS, www.example.com, and you're CURL-ing http? | https? and example.com?

    Can you curl http://www.somewebsite.com? (just pick one).

  • Thanks for getting back to me, Im concerned that it is bypassing the firewall for the http return? is that possible if I see the UDP/DNS/TCP logs in cloudwatch? I posted my stateful/stateless policy below in screenshots

1

Based on the information provided, there are a few potential reasons why you're not seeing HTTP/TLS logs in AWS Network Firewall CloudWatch:

  1. Logging Configuration: By default, AWS Network Firewall only logs traffic that matches drop, reject, and alert action rules. If your rules are set to "pass" for HTTP/TLS traffic, it won't be logged. To log allowed traffic, you need to add an alert action rule before the pass action rule.

  2. Rule Configuration: Ensure that you have stateful rules configured to match HTTP/TLS traffic. Without proper rules, the traffic might pass through without being logged.

  3. Logging Types: Check if you have enabled both Flow and Alert logging. Flow logs capture all traffic passing through the stateful rules engine, while Alert logs capture traffic matching specific alert rules.

To address these issues:

  1. Review your firewall policy and ensure you have rules specifically for HTTP/TLS traffic.

  2. Add alert action rules before your pass rules for HTTP/TLS traffic. For example:

    alert tls $HOME_NET any -> $EXTERNAL_NET any (ssl_state:client_hello; tls.sni; msg:"TLS traffic"; flow:to_server; sid:1000001; rev:1;)

    This will generate logs for TLS traffic before it's allowed.

  3. Verify that both Flow and Alert logging are enabled for your firewall.

  4. Check your CloudWatch log groups. The naming format for log streams should be "/aws/network-firewall/log-type/firewall-name_YYYY-MM-DD-HH" for alert and flow logs.

  5. If you're still not seeing logs, verify that the traffic is actually reaching the Network Firewall. You might need to check your routing configuration to ensure traffic is being directed to the firewall endpoints correctly.

  6. Consider enabling TLS logging if you want to capture specific TLS-related events.

If after implementing these changes you still don't see HTTP/TLS logs, the issue might be with traffic flow rather than logging. In this case, you should verify that your NAT Gateway is correctly configured and that traffic is indeed passing through the Network Firewall before reaching the NAT Gateway.
Sources
AWS Network Firewall logging | AWS re:Post
Troubleshooting logging in AWS Network Firewall - AWS Network Firewall
Logging network traffic from AWS Network Firewall - AWS Network Firewall
Sending AWS Network Firewall logs to Amazon CloudWatch Logs - AWS Network Firewall

profile picture
answered 2 months ago
profile picture
EXPERT
reviewed 2 months ago
  • I have these rules set up so it should alert on all http tls

    alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"Egress HTTP domain"; flow:to_server, established; sid:19361;)
    alert tls $HOME_NET any -> $EXTERNAL_NET any (msg:"Egress TLS domain"; flow:to_server, established; sid:1937231;)
    
    
0

Stateful Policy Stateless Policy Logging

If I do curl http://www.neverssl.com I get these logs

{"firewall_name":"firewall-v2","availability_zone":"us-west-2c","event_timestamp":"1742231528","event":{"dns":{"version":2,"query":[{"type":"query","id":10687,"rrname":"www.neverssl.com","rrtype":"A","tx_id":0,"opcode":0}]},"tx_id":0,"app_proto":"dns","src_ip":"10.12.97.223","src_port":44786,"event_type":"alert","alert":{"severity":3,"signature_id":9999994,"rev":0,"signature":"ALERT: DNS Traffic Logged","action":"allowed","category":""},"flow_id":4784413676689,"dest_ip":"8.8.8.8","proto":"UDP","verdict":{"action":"alert"},"dest_port":53,"pkt_src":"geneve encapsulation","timestamp":"2025-03-17T17:12:08.656473+0000","direction":"to_server"}}

all my suricata rules

# Log all TCP traffic
alert tcp any any -> any any (msg:"ALERT: TCP Traffic Logged"; sid:9999991;)

# Log all UDP traffic
alert udp any any -> any any (msg:"ALERT: UDP Traffic Logged"; sid:9999992;)

# Log all ICMP traffic (ping, etc.)
alert icmp any any -> any any (msg:"ALERT: ICMP Traffic Logged"; sid:9999993;)

# Log all DNS queries
alert dns any any -> any any (msg:"ALERT: DNS Traffic Logged"; sid:9999994;)

# Log all HTTP traffic
alert http any any -> any any (msg:"ALERT: HTTP Traffic Logged"; sid:9999995;)
alert http any any -> any any (msg:"ALERT: HTTP Traffic Detected"; flow:to_server; sid:9999997;)

# Log all TLS (HTTPS) traffic
alert tls any any -> any any (msg:"ALERT: TLS Traffic Logged"; sid:9999996;)

alert tls $HOME_NET any -> any any (tls.sni; content:"www.example.com"; startswith; nocase; endswith; flow:to_server; msg:"TLS SNI alert example"; sid:202501052;)
reject tls $HOME_NET any -> any any (tls.sni; content:"www.example.com"; startswith; nocase; endswith; flow:to_server; msg:"reject example.com"; sid:202501053;)

alert tls $HOME_NET any -> any any (tls.sni; content:".example.com"; startswith; nocase; endswith; flow:to_server; msg:"TLS SNI alert example all"; sid:202501054;)
reject tls $HOME_NET any -> any any (tls.sni; content:".example.com"; startswith; nocase; endswith; flow:to_server; msg:"reject example.com all"; sid:202501055;)
answered 2 months ago
  • I see a big problem: Your ANFW policy is "action order," and it needs to be "strict order." Action order is the legacy method and runs rules in ORDER OF: Pass, Drop, Reject, Alert. And unless carefully crafted, this produces odd behavior. And you also can't put a default action at the end.

    So your policy ought to be Strict Order, Drop Established. You will need to create a new ANFW Policy, and attach it to your ANFW - simple to do!

    The log you're showing - you're using 8.8.8.8 as the DNS resolver? Not the Route53 DNS Resolver? Is that on purpose?

    As far as return traffic, the "Route Table" that contains the Public Subnet, which has your NAT Gateway in it; will obviously have a default 0.0.0.0/0 --> IGW, it will have a local route, but you need to insert a return path (if the CIDR of the private-subnet which contains the workload is 10.10.1.0/24 - your route table needs to have an entry for Destination: 10.10.1.0/24, Next hop: vpce of the ANFW). That will ensure return traffic.

    Once you have validated your return traffic, updated the ANFW policy to Strict Order, Drop Established, Alert Drop established - the try the curl to checkip - this should fail, then curl to http www.example.com & example.com | https www.example.com & example.com - the http should drop with the default action, and https traffic should alert & reject.

  • Wow thank you so much, I was missing the route table needs to have an entry for Destination: 10.10.1.0/24, Next hop: vpce of the ANFW part as well and changed the firewall policy as well -- I appreciate you going back and forth with me on this <3

  • Just to confirm the stateless policy should be to just "forward to stateful rule groups"

  • If for the stateful I do Default actions Alert established Alert all Drop established I cant do any curl (for any url I get curl: (6) Could not resolve host:) But if I remove the "Drop established" at least I can see the HTTP logs -- this way I can be sure I will be able to generate the HTTP domain report (right?)

  • Wow thank you so much, I was missing the route table needs to have an entry for Destination: 10.10.1.0/24, Next hop: vpce of the ANFW part as well and changed the firewall policy as well -- I appreciate you going back and forth with me on this <3 (YOU'RE WELCOME)

    Just to confirm the stateless policy should be to just "forward to stateful rule groups" (CORRECT - no use of stateless engine)

    Default Actions (Strick Order | Drop Established (this means that ANY TRAFFIC not allowed or alerted by rules - will be dropped once it's established) - alert established is more cost effective unless you need all the alert rules in there).

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions