Legitimate traffic can't reach server under DDoS attack

Question

Hi,

We have a gaming server that we are testing in AWS EC2 instance.

Service runs on a custom UDP port, thus we can't use most of the L7 load balancer for protection.

When there is no DDOS we can reach the server and players can play fine and new players can join.
When there is a DDOS, we cannot ping the server anymore although existing SSH sessions work fine and existing players can continue playing but new players cannot join the server anymore.

tcpdump shows that our ping packets are not reaching the server from our source IP, 0 icmp packets from the source IP are found.

Analysis of the DDoS attack shows packets from unique IPs all the time.

We are starting to think that AWS shield is protecting secondary packets, which would explain the uniqueness of IPs, but could also be blocking new connections such as pings and new players.

We do see occasional ping getting through, but only once every 30 seconds for a given IP, the rest are not shown in tcpdump and source ping reports time out.

Is this AWS shield basic taking action on the attack? If so, how to disable it altogether? We have iptables rules in place for protecting but we can't do anything if our packets are dropped before reaching our server.

Answer

The challenge is that identifying what are valid packets vs. DDoS packets is sometimes difficult (particularly if they are coming from different source IP addresses). This is even more difficult with UDP because it is stateless - there are no sessions to track and it becomes harder for automated systems to determine what are "good" vs. "bad" packets.

Using tcpdump is a good start; you might also enable VPC Flow Logs and try to create some NACLs or Security Group rules to drop traffic from known bad sources. While using iptables is good it also consumes CPU resources on your instance(s) - having those packets dropped before they get to the instance(s) is better.

I'd strongly recommend engaging with the AWS support team. They are in a position to perform more in-depth examination and to assist.

Legitimate traffic can't reach server under DDoS attack

Relevant content