[dns-operations] Hall of DNS Shame (?)

[Robert Edmonds]

Ondřej Surý wrote:
> I've been thinking lately (after seeing all the DNS protocol violations)
> that a collaborative list of all DNS protocol violations in the wild
> might be beneficial to both DNS implementors and also to increase a
> pressure on those operators to fix their issues.
> 
> Perhaps we can have such list at some neutral place like DNS-OARC?
> 
> Unfortunately I have a plenty of material I can contribute as of this
> moment.
> 
> My favorite one is log.kmplayer.com that just appends some extra \0
> bytes after the last RRSet in the answer.  (Probably allocating a
> fixed sized buffer and sending it whole instead of just the part
> that was filled?)

I like this idea. The last time I wrote a DNS message parser I ended up
cribbing heavily from the behavior of existing implementations (and what
do you do if two implementations disagree over whether a message is
valid or not?).

It would be cool to have it entirely repository-driven, for example the
repository could have:

 - A collection of broken DNS messages, with annotations like a list of
   references to the RFC clauses violated.

   dnstap would be a good format for storing the raw messages, because
   it doesn't care if the messages are validly formatted or not. If you
   can replicate the problem with a particular kdig command you can
   easily save a trace of the raw query and response messages (plus
   metadata) with something like:

    kdig -E ./dnstap.out +qr [query options]

   E.g.,

    kdig -E ./dnstap.out +qr @61.111.8.236 log.kmplayer.com +norec

 - A utility which extracts the first response message from a dnstap
   trace file and exports the raw DNS message to be consumed by a
   parser.

 - DNS message parser drivers for the various message parsing libraries
   (e.g., libknot, ldns, the BIND9 library, Go miekg/dns, dnsrend).
   There should already be drivers for fuzzing that could be adopted in
   some cases. Probably the drivers can do something simple like reading
   a single wire-format binary DNS message from stdin.

 - A driver script to execute the suite of broken messages against the
   message parsers, and collect and present the results.

 - An audit script which re-executes the collection of broken queries to
   see if the nameservers are still broken.

The whole thing could be automated with GitHub/Travis-CI/etc. If the
presenter script had an HTML mode the output could be saved and
published to GitHub Pages automatically. Then you end up with a nice
"run kdig to generate trace file + add trace file to repository + send
pull request" workflow for adding/evaluating new entries.

That's a fair bit of work but it might be worth it if:

 - There's a ton of broken nameservers out there.

 - Highlighting the broken nameservers might get some of them fixed.

 - Future implementers end up with a high quality collection of broken
   real-world data to test their implementations against.

 - It results in a cool DNS-OARC presentation :-)

-- 
Robert Edmonds