[dns-operations] DNS server benchmarking sanity check

[Anand Buddhdev]

Hi folks,

I'm doing some DNS server benchmarking, and I think I've got a lab setup
that looks right, but I'd like to run my config past you guys, the
experts, to make sure I'm not overlooking anything obvious.

I have 3 Dell R320 servers, each with a single Intel Xeon E5-2470
10-core processor at 2.40 GHz. The OS therefore sees 20 processors. The
server has the usual on-board dual-port 1 Gbit ethernet ports, but I'm
not using these for the testing. These are merely for logging in and OS
management. The servers also have dual-port 10 GB NICs, with an intel
chipset, and using the Linux "ixgbe" driver.

One 10 GB NIC in each server is connected to a switch, so that the 3
NICs are in the same LAN.

The OS is an up-to-date CentOS 7 installation. The roles are thus:

1. server 1 is the source of packets. On it, I have compiled the latest
tcpreplay with the --enable-quick-tx option, to allow it to play back
packets at a high rate. I have a pcap file containing 6,882,162 real DNS
queries (UDP-only) which I can send out to a DNS server.

2. server 2 is the DNS server. I have so far installed BIND 9.10.4-P1,
NSD 4.1.10, Knot 2.3.0 and Yadifa 2.1.6 on it. They are all configured
as root name servers, with the root, arpa and root-servers.net zones on
them.

I have done minimal tuning of this server. I have set these 4 sysctl
settings:

net.core.rmem_default=4194304
net.core.rmem_max=4194304
net.core.wmem_default=4194304
net.core.wmem_max=4194304

Additionally, I've configured the following iptables rules on it:

-A PREROUTING -i p1p1 -p udp -m udp --dport 53 -j NOTRACK
-A OUTPUT -o p1p1 -p udp -m udp --sport 53 -j NOTRACK

With these rules, I ensure that no connection-tracking is done, and I
can count the number of incoming packets (queries) and the number of
outgoing packets (responses) easily.

Finally, there is a default route installed on the server to send all
the responses out to server 3.

3. Server 3 is just the sink. It received DNS responses, and there's a
single iptables rules to drop them all (and count them):

-A PREROUTING -s 193.0.14.129/32 -p udp -m udp --sport 53 -j DROP


With this setup, I can emit a DNS query from the source, and it goes to
the DNS server, which emits a response, and the response goes to the
sink where it is discarded. On the source, tcpreplay is able to achieve
a maximum packet rate of just over 3 million, so I can blast over 3
million q/s towards the DNS server. At that rate, none of the DNS
servers can keep up of course. However, I'm playing back packets are
various rates to find the breaking points of the various servers.

Now I know that there is a LOT more tweaking and tuning that can be
done. For example the 10 GB NICs have lots of tunables. In addition,
there are things like CPU affinity, IRQ affinity, etc that could be
tweaked. But let's assume that we don't want to get into that level of
tweaking and tuning, and just want to use the defaults of the OS with
minimal adjustment, is this setup okay? Are there any other Linux kernel
adjustments, especially on the DNS server, that should be adjusted for
high packet rates?

Regards,
Anand Buddhdev
RIPE NCC