[dns-operations] dnsflow again (Re: DNS Traffic Archive Protocol)

[Jason Chambers]

On 12/8/10 5:16 PM, Paul Vixie wrote:
>> you would then have a bunch of messages with the smaller tuples:
>>
>>     (type, query_ip, response_ip, proto, qname, qtype, qclass, rcode)
> 
> i don't want deletion or reduction, i'm looking for metrics made up of
> key:value pairs where the "key" is a compound like "how many times did host
> X perform action Y" and "value" is a counter.  this isn't just lossy, it's
> transformative.  you could see a million dnsqr's per second on input and
> generate a hundred dnsflow's per second on output, depending on how self-
> similar the inputs were.  for that matter you could see a thousand dnsqr's
> per second on input and get tens of thousands of dnsflow's per second on
> output if the self-similarity of the input was low enough.
> 

What your discussing is very similar to the SiLK toolset for flow
analysis.  Glad to see others are working on this.  The IPFIX idea is
very interesting as I had not considered that.  For some time now I've
been thinking how to modify the SiLK [1] toolset to include DNS data and
leverage useful analytical tools like rwuniq [2] and rwcut [3].  Using
IPFIX would make that much easier; up until now I was considering a
custom format.  Sorry I don't have any working code at this point; it's
been a lower priority project for some time now.

For those who are not familiar with SiLK, please take a look at it for
influence because a lot of what has been said is very similar to the
SiLK toolset and a review might save hours or days of brainstorming.

[1] http://tools.netsa.cert.org/silk/faq.html#what-silk
[2] http://tools.netsa.cert.org/silk/rwuniq.html
[3] http://tools.netsa.cert.org/silk/rwcut.html


Regards,

-- 

Jason Chambers
UCLA
jchambers at ucla.edu
310-206-5603