Suricata rules not working consistently.

There are a few potential reasons why your Suricata rule for www.bbc.co.uk isn't working as expected while the Google rule works:

1. Fragmented Client Hello: The most likely issue is that the TLS Client Hello message for www.bbc.co.uk might be fragmented across multiple packets. When this happens, Suricata may not be able to properly inspect the SNI field. To address this, you should add the ssl_state:client_hello keyword to your rules:

pass tls $HOME_NET any -> $EXTERNAL_NET 443 (tls.sni; content:"www.bbc.co.uk"; endswith; msg:"Allow BBC"; flow:established; ssl_state:client_hello; sid:1003; rev:1;)

2. Rule Evaluation Order: Even though you're using strict ordering, there might be issues with how the rules are being evaluated. The stateful rule engine evaluates rules based on the lower-level protocol before the application layer protocol, which could affect your rule matching.

3. Rule Syntax: Double-check for any typos or syntax errors in your rule for www.bbc.co.uk. Even small differences can cause rules to fail.

4. Traffic Flow: Verify that the traffic to www.bbc.co.uk is actually flowing through the Network Firewall and that your HOME_NET and EXTERNAL_NET variables are correctly defined for your network architecture.

5. Consider using "reject" instead of "drop": Some troubleshooting scenarios suggest using "reject" instead of "drop" to see if that makes a difference in rule behavior.

The most effective solution is likely adding the ssl_state:client_hello keyword to your rules to handle fragmented TLS Client Hello messages, which is a common scenario with modern TLS implementations.
Sources
AWS Network Firewall : Suricata to filter on tls protocol for specific domain doesn't work | AWS re:Post
AWS Network Firewall enhances application layer traffic controls - AWS

I think I've worked out what's happenning and it seems to be something quite specific to the way certain CDNs are handling the TCP negotiation. If you look at a TCP handshake with www.bbc.co.uk, the SYN-ACK packet from BBC consistently sets the TCP Window size to 0. This means the client, after sending its ACK packet needs to wait for a TCP Window Update from the BBC before proceeding with the session and this is exactly what happens if not blocked by a firewall. So every conversation with www.bbc.co.uk follows this pattern: -

1. Client sends SYN
2. BBC sends SYN-ACK with Window set to 0
3. Client sends ACK
4. BBC sends TCP Window Update with an updated TCP window size and scaling factor
5. Client proceeds with sending TLS Client Hello.

So with our current Suricata rule set, there isn't a rule that allows the TCP Window Update from the BBC to pass back to the client, as the packet is now part of an established TCP session, but obviously isn't handled by any of our TLS specific rules. A packet capture on the client side shows the SYN, SYN-ACK, ACK sequence but nothing further other than the client repeatedly sending the final ACK packet. The client can't begin initiating the TLS Client Hello because as far as it knows, the server isn't ready to receive any packets at all as the last packet it received from the server - i.e. the SYN-ACK - set the TCP Window = 0.

The quick "fix" in this case is to have a rule that allows the TCP Window update to pass, i.e.: -

# Allow a TCP packet within an established flow from the server back to the client
pass tcp $EXTERNAL_NET 443 -> $HOME_NET any (flow: established; sid:1002; rev:1;)

Implementing this rule does indeed allow for a successful connection with www.bbc.co.uk. But this isn't great, as it doesn't allow for non-TLS but established TCP messages to pass from the client to the server - e.g. TCP Window Updates from the client will also get blocked. And we can't implement the reverse rule, i.e.: -

pass tcp $HOME_NET any -> $EXTERNAL_NET 443 (flow: established; sid:1003; rev:1;)

...as this will basically allow all TCP port 443 traffic to anywhere. i.e. our subsequent TLS SNI based rules are superfluous as the packet has already been passed by the firewall at the TCP layer.

The correct fix therefore is as per @mleighty 's link to the AWS Security Best Practices article, which advises to avoid using firewall policy default stateful actions, and to include these as explicit rules in a Suricata ruleset.