[dns-operations] compressing DNS traffic data

[Elmar K. Bins]

Re Ondrej,

ondrej.sury at nic.cz (Ond??ej Surý) wrote:

> 1. cost per unit are the same when you start piling them up.  It's not
> just cost of manhours needed to maintain the storage, it's also the
> electricity, cooling, cost per rack space, interconnections.
> 
> 2. we need to waste it even if it's cheap.  It's the same flawed logic
> why the US cars had such low mpg ratio - because gas was cheap there was
> no "need for efficency".  If the "compression" ratio is 1:5 or 1:10
> compared to raw packets, it will pay of now and in the future.

I agree, and I even want to add something: Most of the time, you can
easily get away with sampling the data. Networkers are pretty much
used to their sampled netflows, and on a broad, statistically relevant
basis, you lose not much of significance, but compress your data stream
significantly. So I'm all for adding "sampling" to the feature set.


> Robert, I have a feeling that we are steering towards "one size fits all" and I 
> don't think we can fit all requirements in one big protocal.  Obviously your 
> requirements are different than ours (and that's not really a bad thing) and I don't 
> mind developing different (not necessarily competing) protocols for different needs.

The results everybody wants to have, are similar, but not identical.
Granted.

Having a unified/standardised data storage format nonetheless greatly
helps: Developers need not invent new formats and the data dumping
software that goes with them, but can concentrate on using the data,
analysing and visualising it. As long as it's not XML based, I am all
for a standardised (and extensible) data format ;)

Yours,
	Elmar.

-- 

"Machen Sie sich erst einmal unbeliebt. Dann werden Sie auch ernstgenommen."
							     (Konrad Adenauer)

--------------------------------------------------------------[ ELMI-RIPE ]---

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
URL: <https://lists.dns-oarc.net/pipermail/dns-operations/attachments/20101208/c50d7e44/attachment.sig>