DNS DDoS - let's use a long term solution
Last Updated: 2009-01-31 17:17:56 UTC
by Swa Frantzen (Version: 2)
The current batch of DDoS attacks continues now for quite a few days. Let's recap a few things and look forward to a long-term solution.
The current attacks:
A spoofed UDP query causes a reply of the root cache information which is significantly larger (~330 bytes) than the query.
The victims are twofold: those who reply to the query (or even just get probed for it and not reply) and those who are the ultimate victims of the DDoS (yes those who reply to it are victims, nothing more).
This results in two things: the botnet of the attackers is harder to trace back by the ISPs involved as the spoofed packets are each fairly small (60 bytes) and do not come near the ultimate victim. So the white hat community needs to work more closely and harder to track down the source(s). Also the amplification factor is significant (~5.5x) which means that the attackers can use less of their bots to completely flood the ultimate victim.
Quite a few calls are being made to stop giving out root cache copies, some even going as far as calling those who operate the intermediate victims incompetent and worse.
While I fully sympathize with the ultimate victims, I also sympathize with the intermediate victims as they've not done that much wrong, In fact, it can be argued they did absolutely nothing wrong.
In the past we have seen similar attacks. They also had large amplification factors. They used open resolvers to query a very large TXT records from a DNS server and those resolvers sent those huge (cached) replies back to the victim who's ip address had been spoofed.
The call at the time was for us all to stop having open resolvers, something that originally was considered being neighborly, friendly became a network hazard and in fact offensive to continue to offer.
Some reading from 2006 on these attacks:
- ICANN: http://www.icann.org/en/committees/security/dns-ddos-advisory-31mar06.pdf
- US-CERT: http://www.us-cert.gov/reading_room/DNS-recursion033006.pdf
- RFC-5358: ftp://ftp.rfc-editor.org/in-notes/rfc5358.txt (Oct 2008, but on this subject)
Longer term solutions
Clearly attacks evolve (even if it takes a few years), so we need to be ready for the next ploy by the time the attackers hit us with it.
What's to stop them once we shut down enough root caches responders to start asking our name servers questions we have to answer? Something like a large reply in a domain the server is authoritative for? Sure, not all servers will need to answer the same query, but all things considered that is not very complex for a botnet controller to cope with: have a table of who get's which query takes no rocket scientist to program.
So what are we going to shut down when they do this? Are we going to start to hunt for the long replies that people might have and try to make them shorter?
Or are we finally going to put pressure on the ISPs to stop once and for all the ability of their customers to spoof their source IP address.
The root problem isn't so much a stateless protocol like UDP replying to something with a longer reply than the question. In itself it's not a problem as long as IP spoofing is made impossible.
So what anti-spoofing measures are we talking about?
Is stopping spoofing at the AS borders enough? Not really: it still allows the bad guys to group their botnets per ISP (if they haven't done so already), send their spoofed requests within the ISP and then have a non-spoofed reply to it go to the victim. Moreover this can't be done at transit providers borders as it would greatly impact the self-healing feature of the internet.
Is stopping the spoofing at the border between the ISP and the individual customer enough? Bingo! But these filters aren't trivial to implement:
- Regular dial-up and even xDSL and cable customers get a dynamic IP address, forcing the filter to be dynamic as well.
- Some larger customers are mixed in with the consumers but have routed networks, forcing the complexity of the dynamics to actually adapt the filter from the routing tables
- Some customers are multi-homed. They have connections to multiple ISPs and want the ability to send out packets to one ISP even when they'd get the reply on their other connection. Depending on just how this is done (multiple options exist), this can require allowing the addresses the other ISP(s) have allocated to the customer, using information from the ASN of the customer etc.
So full ingress/egress filtering isn't easy to achieve and vendor's equipment in active use might not even be able to support it. IMHO it's the only thing that will make all stateless protocols safer from being abused to either to amplify the attack, or hide the real location of the attackers. Some botnets out there are by far large enough to blow just about anybody out of the water, amplification is not needed as such by one of the larger botnets (do the math of upstream capacity times number of bots).
- BCP 38, RFC 3704 best current practices regarding ingress filtering dating back to May 2000 and March 2004 respectively.
- Unicast Reverse Path Forwarding: Aimed at ISPs; general understanding, a "cheap" way to link routing info in allowing traffic in the reverse direction.
So in the end, it's my opinion that pressure needs to be put on those ISPs that do not have full anti-spoofing measures for all their customers.
Now if you run a sizable network, you can help with this too: prevent all source addresses that aren't in your assigned official address space from leaving your network onto the Internet. You won't filter away valid traffic as you can't get answers anyway and you're doing your good deed (hint: log the traffic, it might point to misconfigured and/or infected hosts).
If you're an intermediate victim, please do not see this text as an excuse not to help minimize the ongoing attacks by removing long root cache replies. You're in a position to help (as little as each of you can individually), please do so even if you're not the root cause of the problem.
Swa Frantzen -- Section 66
It's simply too complex to implement properly to work: it's much cheaper for the ISP to deal with the consequences (i.e. slight traffic increase unless they are on the receiving end) than fix it. In consequence, it will never be fixed for any meaningful portion of the IPv4 space.
The real problem here is DNS. That dinosaur of a protocol is everywhere: there isn't a single IPv4 machine that doesn't understand it, DNS servers are numerous and (relatively) arcane to configure correctly, the infrastructure is waaay to fogiving to improperly configured servers and the core protocol is stateless. I can't think of a single example of a more dangerous protocol out there: even SMTP is better.
A short term solution would be to force all queries to go through TCP instead of UDP. It really shouldn't be a problem: all DNS servers out there have the capacity to do that already and we have a central point where that can be enforced: change the root servers to allow for TCP queries only and you're going to force everyone to update. Sure, people will still have the option to configure their servers incorrectly afterward but they WILL have to do so willingly. After this, ISP could start blocking UDP 53 traffic, both way and so could DDOS victims: there will be no harm done since "well behaved" servers would now used a different port.
The only really trouble, of course, if whether the root servers can handle this or not. I frankly have no idea, but if they can't, then it'll certainly be cheaper to update that infrastructure than any other proposed solution.
Feb 2nd 2009
1 decade ago
Shutting down DNS over UDP is costly on servers, and will slow down all Internet traffic, not something ISPs will want to advertise.
I'm not saying it's easy, but it is possible and some ISPs are doing it properly.
DNS isn't the problem, spoofing is. And spoofing is a problem for more than just DNS or even UDP.
Feb 9th 2009
1 decade ago