[dns-operations] DNS server benchmarking sanity check

[Jared Mauch]

> On Aug 15, 2016, at 3:00 PM, Florian Weimer <fw at deneb.enyo.de> wrote:
> 
> * Anand Buddhdev:
> 
>> On my hardware (10 cpus, visible as 20 to the OS), running nsd with 20
>> worker processes and SO_REUSEPORT, it is able to answer 100% of the
>> queries at 1.2 million q/s, and 97% of the queries at 1.6 million q/s.
>> By this point, all 20 cpus are at 100% utilisation, so there's no way I
>> can get anything better, and performance drops with increasing query
>> rates. The only thing to do is to get faster cpus.
> 
> You could make sure that most time is spent in user space, and if not
> look at perf output (on Linux), check to make sure that packets are
> received on the CPU which later handles them, and so on.
> 
> I can ask around, but I assume it is possible to saturate 10G on such
> hardware.

One thing I’m always reminding people is that most OS’es have optimized
for TCP performance and not UDP.

50Gb/s for TCP ~6Gb/s for UDP.  I’ve been frustrated by these defaults
in Linux that result in such a performance difference as the outcome.

eg:

puck:~$ iperf -c localhost
------------------------------------------------------------
Client connecting to localhost, TCP port 5001
TCP window size: 2.50 MByte (default)
------------------------------------------------------------
[  3] local 127.0.0.1 port 60088 connected with 127.0.0.1 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  58.2 GBytes  50.0 Gbits/sec
puck:~$ iperf -c localhost -u -b 50000m
------------------------------------------------------------
Client connecting to localhost, UDP port 5001
Sending 1470 byte datagrams, IPG target: 0.24 us (kalman adjust)
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  3] local 127.0.0.1 port 43066 connected with 127.0.0.1 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  6.78 GBytes  5.82 Gbits/sec
[  3] Sent 4950578 datagrams
[  3] Server Report:
[  3]  0.0-10.0 sec  6.58 GBytes  5.66 Gbits/sec   0.000 ms 141142/4950578 (2.9%)


Moving from using send() to sendmsg() can help performance as well as
other tuning, but these tend to be less portable and avoided by developers
in my experience.  Simply put, this is the one case where I would still use
FreeBSD > Linux.  Other than that, my personal pains w/ FreeBSD caused me
to leave their ecosystem.

- Jared