[dns-operations] dnsflow again (Re: DNS Traffic Archive Protocol )

[Robert Edmonds]

Beda Kosata wrote:
> Are we the still talking about the original dnsqr schema or is this
> something new, similar to what Paul proposed? To me it seems that
> with so many adjustments, you have just laid out plans for a
> completely new schema.

i'm talking about modifying the existing dnsqr schema to be more
flexible and accomodate applications like this, not the creation of a
new schema.  what i've described doesn't exist (yet) but if it's the
right approach i'll be happy to implement it.

i also want to do this inside the dnsqr schema because i'm developing a
boolean expression filter language for dnsqr messages and i'd prefer to
have a unified DNS filtering language rather than having to develop two
in parallel.

> BTW, I did some preliminary tests and I think that a nmsg schema
> which would resemble completely or almost completely parsed DNS
> response (with most parts optional) could be a good way to proceed.
> It would allow different parts of the data to be omitted, which
> seems crucial for such a format, but could also accommodate all the
> data and in all cases allow for faster parsing - due to elimination
> of different network layers, fragmentation, ip versions, etc.
> I tried converting data from my experimental format into a protobuf
> based format. The result was almost twice the size, but it
> compressed to almost the same size when advanced compression (xz)
> was used. Therefore it seems to me that nmsg is really the way
> forward because tools are already available to work with it.

interesting.  how does xz compare to gzip in terms of CPU time?  NMSG
already has support for transparent zlib compression (nmsgtool -z), it
would be easy to add liblzma support.

i think this is veering off-topic for dns-ops.  mail followups to
nmsg-dev at lists.isc.org, please.

-- 
Robert Edmonds
edmonds at isc.org