[dns-operations] dns-operations at lists.dns-oarc.net

Wed May 9 14:47:23 UTC 2012

If you are looking for DDoS resilience, the answer is not "X times normal".  A DDoS is not a multiple of your normal traffic, it is whatever the botnet can throw at you.

This is obviously different for everyone.  If you are a small provider with a couple GigEs of transit, then having capacity for billions of qps is silly.  If you are a vital part of the Internet (roots, GTLDs, or even CCTLDs, Google, etc.), then having the ability to answer 10s of Gbps of queries is important.

DNS queries are easy to spoof, they don't require two way communication[*] making it difficult[**] to tell spoofed from real packets.  Many providers have decided the best defense is to simply answer all the queries.  Unfortunately, this can result in the spoofed source being hit (see all the threads about amplification).

Anyway, the point of "size for X times" is fine for normal operation.  But for DDoS, normal traffic is irrelevant.  Your upstream / peering pipes are, and the amount of money you want to spend is, nothing else.  Oh, and how much protection your customers expect, but that devolves into "how much you want to spend". :)

-- 
TTFN,
patrick

[*] Please don't bring up the cisco guard or whatever it was that set the TCP bit.  It breaks many things horribly badly.

[**] The ability to detect spoofed packets is difficult and sometimes not possible, but that's not a discussion for an open list.

On May 9, 2012, at 10:29 , Stephane Handfield wrote:

> Hello Todd,
> 
> It's exactly the X times of the average qps rates I'm looking for, on our side, we try to do a capacity planning for recursive/caching DNS which are accessible only by our clients, no firewall on the path, only acl's at the edge for protecting the cache DNS of the internet but we're using a mix of LoadBalancers and anycast, which make the LB's a possible choke point. At that point, we define the following rule,  take the estimated QPS we'll have to support at the end of the year and put in place enought hardware to support that qps rate by 10x ... but 10x is only a number everybody here seem to be confortable with.
> 
> Thanks,
> 
> Stef
> 
> On Wed, May 9, 2012 at 9:10 AM, Todd Snyder <todd at borked.ca> wrote:
> I want to know what rules you follow in terms of capacity planning for your DNS. I am mainly interested in the best planning practice for caching  DNS. Definitly our rules need to reflect a lots of our own reality, in term of agility of deployment, risks, etc. But I'm really interested to know if exists any rule of thumb which mostly apply to any situations.
> 
> 
> I've just finished a capacity exercise, and followed through with new hardware.  We decided that in our business (a service provider, kind of), our externally facing authoritative servers *must* be able to handle 100x our normal production query rate.  It *should* be able to handle 300x production query rate.  We ended up with a service that, due to our topology and desire for redundancy, can handle 700-800 times our normal query load.  We also have a plan in place to double that within minutes should it be required.
> 
> This is all in place in case anyone ever decides to DOS our DNS service - we want to make sure that we have the most capacity available, striking a balance with cost and operational efficiency.  Without DNS, our services and customers are off the air, so it was deemed to be worth the resource outlay to protect that service.  It may not be as important to others.
> 
> Something to note is that DNS packets are small, and plentiful.  Those attributes make firewalls go *kaboom*.  If you're looking at how to build a DNS service, don't look at just the DNS server capacity, but the firewall (and other network pieces) you (may) put in front of it.  We removed the firewalls for just this reason.  It's great that you can build a DNS server that can handle X qps, but when your firewall fails at (X/10)qps, you're in trouble.
> 
> I don't know if this helps, I don't even know if those numbers make sense to other operators.  It's very difficult to get people to talk about this stuff in real, operational terms.
> 
> Cheers,
> 
> Todd.
> 
> _______________________________________________
> dns-operations mailing list
> dns-operations at lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations
> dns-jobs mailing list
> https://lists.dns-oarc.net/mailman/listinfo/dns-jobs