[dns-operations] Should medium-sized companies run their own recursive resolver?

[Jared Mauch]

On Oct 16, 2013, at 6:39 PM, Vernon Schryver <vjs at rhyolite.com> wrote:

>> From: Jared Mauch <jared at puck.nether.net>
> 
>> Understanding how this works is not networking or DNS 101. Limiting
>> the scope with TTL isn't that easy.
>> 
>> Can you point someone at docs for how to do that in a point and click fashion?
> 
> Can you address the issues instead of dragging in irrelevancies?
> 
> The operating system on that hypothetical, as yet non-existent "DNS
> appliance" that could use the TTL to keep it from being an open resolver
> might need a new setsockopt or socket ioctl to set a per socket TTL.
> You have my opinion as someone who has done such things professionally
> that in at least the BSD stack, such a change, if necessary, would be
> trivial.  I strongly suspect that would also be trivial in the Linux
> code.  That assumes that your DNS server code does not use other,
> existing but less application-programmer friendly mechanisms.

I think the difference is this is an -operations list, so I'm looking at/around things
that can be done to operate the equipment.

> GUI pointing and clicking to maintain a suitable stanza into a DNS
> server text configuration file would be almost as trivial.

This certainly can be true, but the average operator isn't going to understand
that IP TTL != DNS TTL, and may not even be aware of that part of the packet.

The average user is going to use a document like this to configure their DNS:

https://support.google.com/a/answer/48090

> } From: Jared Mauch <jared at puck.nether.net>
> 
> } Comcast doesn't give me broken name servers to use, there is no cognitive dissonance here :-)
> 
> That statement asserts facts not in evidence.
> 
> } You are a DNS expert. Most end users when DNS fails think everything has failed, including the network.
> }
> } I type URLs into my browser. Do you know how many people type google into the google search box? Or the yahoo box?
> }
> 
> Yes, and as I wrote, it is unrealistic, unnecessary, and wrong to
> expect users such as the IT professionals in that 2-person department
> to determine whether an ISP DNS servers is broken.  It should be
> realistic and should be required that one of them to be able to determine
> whether BIND or NSD on a whitebox is working.  The intentional breakage
> of ISP DNS servers is too subtle.  It's not merely NXDOMAIN rewriting
> but other craziness like AAAA filtering for IPv4 clients.  (Again, in
> the context of consumer households, that crazy breakage might be good
> and even necessary.)
> 
> } You seem disconnected from the average user and average user tech support.
> 
> Why do you always descend to ad hominem?

I think because I'm frustrated with part of premise around the discussion.

Most of these "advanced" DNS things like RRL, RPZ and others aren't for
the faint of heart.  Most people don't watch/monitor logs like those here.

I can't even get my vendors to fix their software bugs after months of saying
"it breaks when I do X", and I pay you $X mm/year to service these, including
the software updates.  Even for those advanced in the space, these things
are difficult, unclear and fragile at best.

>  I have some experience with
> "average user tech support," thank you very much.  Your mail was to
> me delayed while I wrestled with CenturyLink's Asian (he refused to
> be more specific, but I heard Hindi when he got confused with his mute
> button and in the background when his squelch hiccuped) "average user
> tech support" that wanted me to "reboot the modem" and tell him the
> version of Windows on "the" computer.  (After the third time I blew
> up, he started listening and called someone who helped him, so that
> we ended the call with my DSL carrier restored and the ticket closed.)
> 
> Why do you insist on talking about irrelevant scenarios involving end
> users?  Neither of the people in that 2 persion IT department would
> admit being average end users.  Of course they would not know as much
> about DNS as they might, but they could understand that hypothetical
> DNS appliance at least as well as their LDAP, AD, HTTP, and SQL servers.
> 
> More important, why do you ignore my point about required minimum
> competence?  Long ago, you could buy an airplane and go into business
> with it without getting permission.  Not so long ago, you could buy
> or lease 5 acres and grow olives, melons, or corn with no worries about
> licenses or going to jail if an animal happens to defecate upstream.
> Neither is possible today.  (In at least Calif you need state health
> inspections and licenses to sell your own olive oil.)  You don't need
> a degree in aeronautical engineering to be a pilot or in communicable
> diseases to farm, but you must demonstrate minimal competence.  It
> should also not be possible to get a job in the 2 person IT department
> at issue without understanding enough about DNS to install and maintain
> a white box run a simple BIND, NSD, or other recursive DNS server.

I think the challenge here is that there is no true certifying authority for
the industry.  You speak of state or possible federal (or transnational licenses)
regimes of inspection, licensing, authorities, etc.  They don't exist here.  There
are standards, and even "RFCs", but if I operate a service on UDP/53 that
doesn't do what those RFCs state, there is truly no recourse.

The devices doing DNS these days aren't BIND, NSD or others.

The software provided by Microsoft which is most likely to be deployed in the LDAP/AD environment can't be properly secured to operate safely on the internet.
> 
> } Even small networks (I have a friend with a ~100 user wisp) shouldn't run their own caches. The economics of it don't support this.
> 
> "Economics" in this century have nothing to do with where and when
> local DNS caches are good or bad, necessary or useless.

I think that's the point of the discussion though, "Should medium-sized companies run their own recursive resolver".  100 subscribers, with possibly 500+ devices behind it (n*iPhone+n*iPad+n*Android+n*TV+n*Appliance) are common these days.  You can easily assume that each person has at least *one* device,with average household sizes, etc.. and you're in the right range for 4-5+ devices per home at least.  So 500 unique stub resolvers is a large population, but at the same time, it's not enough to justify spending $1k for a server *2 + UPS.  That cost on my friends WISP would wipe some or all profits.  Perhaps he should charge more, but he doesn't have the knowledge or skill to operate things, but does know it might help to have a faster box on-net…

> I am offended on behalf of those hypothetical IT professionals by your
> persistent infantilizing them.  Attitudes like yours in ISPs are why
> there there is so little BCP38 compliance and so many open resolvers.

Ha!  BCP-38 is actually *hard*.  Customers don't know their IP ranges or domains they own/operate.

*really* they don't.  I know this because of time spent trying to sell a security service to folks who wanted it.  They couldn't even enumerate their assets/resources, and these were not all *small* places either. 

> If ISPs would refuse to route packets to customers that can't comply
> with BCP38 or that run unnecessary open resolvers or open resolvers
> unprotected by rate limiting, then a lot of problems would go away.

I encourage you to start an ISP and report back your results :)

I can either forward packets or filter them in many cases.  The damaging packets, even when there are large attacks are still such a small percentage of the overall traffic that I can't sacrifice them for the sake of others.  You've seen other providers back away from BCP-38 over the years.  I continue to strive in this direction and I resent the fact that you think we're all not trying to do it.  If you think you can build a better router that can do it right, please do.  The market needs it, but that's OT for this list.

- Jared  (really, it's not personal, maybe i'm just frustrated with the state of the industry across the board right now)