Identifying root-cause for packet drops

Hello,

I end up with trying to delete / add rules one by one to detect what is causing the problem.

Referring to the above comment, what is the goal here? Are we trying to check if the packets get dropped because of security groups or network ACLs? If yes, you can use VPC flow logs to identify the rejected packets. [1]

Also any traffic rejected on the OS level will not be captured by the VPC Flow logs.

[1] https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs-records-examples.html#flow-log-example-security-groups

[2] Publish flow logs to CloudWatch Logs - https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs-cwl.html

[3] Logging IP traffic using VPC Flow Logs - Available fields - https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs.html#flow-logs-fields

You can use AWS Reachability Analyzer tool to determine cause of issue between source and destination due to Security or NACL. It also include additional path components that gives you more information on packet drops. https://docs.aws.amazon.com/vpc/latest/reachability/what-is-reachability-analyzer.html

Oh man, I feel you on this one. Troubleshooting connectivity issues without per-rule counters is like finding a needle in a haystack. 🌾📌

I've been in a similar spot, and you're right, deleting and adding rules one by one is beyond inefficient; it's like playing a game of whack-a-mole. 🐾🔨

If you're working in an AWS environment, they've got VPC Flow Logs that can give you some data, but it's not as straightforward as 'iptables -nvL INPUT --line-numbers'. Unfortunately, as far as I know, most cloud providers don't give you per-rule match counters like a traditional Linux box. 😭☁️

For now, you might want to consider third-party tools or network monitoring solutions that can give you more granular data, but yeah, it's a gap in the feature set that I wish would get addressed soon. 🧐🔍

[1] https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs-records-examples.html#flow-log-example-security-groups-yacine

[2] Publish flow logs to CloudWatch Logs - https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs-cwl.html