[dns-operations] DNS measurement standards

[Mike Hoskins (michoski)]

I've used dnsperf in the past.  Liked the company's reputation, the design of the tool, and the ease of getting visual output.  I haven't used the others, so none of this is to say these things aren't also true about them...just what I liked about dnsperf.

The queryfile format makes a lot of sense, and is great for testing real-world performance...  It's easy to use the one they provide or build your own from tcpdump/query logs.  I usually do several runs with both cold and warm caches and take averages to find interesting edge cases.  That said, you are ultimately "testing the Internet" with this approach...and as others have pointed out much of the "performance" is responsiveness of authoritative servers for zones in your queryfile.  For me, this was an interesting thing to test since it finds many of the same issues seen in running DNS servers day-to-day.

Having said all that, in terms of an approach to measurement...if I was more concerned about raw throughput of a specific name server software, I would be sure to setup a lab so that I could control "the Internet" and "remote server responsiveness" -- similar to what I'd imagine product stress testers do in their labs.  Build a query file with N random internal domains delegated to a farm of internal NS over a fast LAN...then run batches of queries with drop rate capped at some reasonable percent (I usually do 1% which is a lot better than what's generally observed in the real world due to upstream problems).  This qps number will obviously be a lot larger since you are now closer to comparing internal vs end-to-end performance.

From: Kumar Ashutosh <askuma at microsoft.com<mailto:askuma at microsoft.com>>
Date: Monday, December 8, 2014 at 10:34 AM
To: "dns-operations at dns-oarc.net<mailto:dns-operations at dns-oarc.net>" <dns-operations at dns-oarc.net<mailto:dns-operations at dns-oarc.net>>
Subject: [dns-operations] DNS measurement standards

Hi
This may be a repeated question, but hoping a quick reply on this. There are multiple tools to measure DNS server performance. And there are umpteen ways to measure as well.
I have come across many tools like ISC queryperf, nominum’s DNSperf, Dnsbench etc…

I may start a debate, if I ask which one is the best. So let me ask which *are* the most popular and Is there a standard take on the way to measure.

Regards
Kumar Ashutosh
Microsoft

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.dns-oarc.net/pipermail/dns-operations/attachments/20141208/43843cde/attachment.html>