[dns-operations] BIND behaviour when using multiple active ZSK's

[Thomas Dupas]

Hello All,

we've stumbled upon a particularity with Bind 9.8 and 9.9 versions, and are wondering if anyone else already bumped into this.
When using more than 1 active ZSK, in a bind auto-maintain scenario, BIND tends to lose control when renewing RRSIG’s.
You experience this as sudden bursts in serial increments / NOTIFY's.
When you raise the logging level you see that it renews some RRsigs over and over again (several thousand times).

We are able to reproduce this on:
- various BIND 9.8 and 9.9 versions. Not on BIND 9.7 versions (haven’t tried 10 versions)
- pre-packaged versions, hand compiled versions with both -O0 and -03
- both physical / hardware virtualized and para-virtualized hosts
- various linux distributions and kernel versions (tested EL5/EL6 variants and debian variants)
- both production zones as test dummy zones
- with NSEC, NSEC3 and NSEC3 opt-out setups
- with various timings / different DNSKEY's

The likelyhood of this happening increases when the amount of pending RRSIG's increases, we do get a ~100% reproduction rate when using the following setup:
- dummy zone with 1M records, each with 2 nameservers and 1 DS record
- sign it with an expiration of 7 days in the future, and a 600sec jitter interval (it also happens when using a jitter interval of 1 hour, but it was reduced to facilitate testing)
- fast-forward little over 5 days (aim a little earlier than the 75% interval when BIND will renew signatures)

cat debug7.nfo | egrep "add re-sign" | awk '{ print $8 }' | sort | uniq -c | sort -rn -k 1 | head -n 20
 236367 eu.
   5654 GI0AO9KPOOHS3HQCRAUR3ADT5QMR94RT.eu.
   5471 testdomain-911794.eu.
   5178 testdomain-389749.eu.
   5077 testdomain-199411.eu.
   5019 testdomain-387060.eu.
   4881 J11CL0B2DNTMFI3UPD5KS8PC7GNCDI58.eu.
   4711 17IJ2OFH012BBAN78FVLGQ11Q37J0N6E.eu.
   4562 CHKMSCN61P7NLDC5JQ7APVGPJVJRFRLR.eu.
   4417 85B9IPB80VJIE6IKPE4KU2FBKRR71MM3.eu.
   4247 testdomain-461370.eu.
   4124 J0G8D50KAPM787DSVREK9S32CR8KG9HO.eu.
      1 testdomain-999999.eu.
      1 testdomain-999998.eu.
      1 testdomain-999997.eu.
      1 testdomain-999996.eu.
      1 testdomain-999995.eu.
      1 testdomain-999994.eu.
      1 testdomain-999993.eu.
      1 testdomain-999992.eu.

If you go through the log for one of those RRset's you will see that it continues to remove/generate signatures over and over again for several minutes, and than it suddenly stops for that RRset.
A bit later you might however see the same behaviour for another RRset

Br,

Thomas Dupas
EURid vzw.		http://www.EURid.eu
The European Registry of Internet Domain Names

Disclaimer:
This email  and  any  attachment  hereto  is  intended  solely  for  the  person
to which  it  is  addressed  and  may  contain  confidential  and/or  privileged
information.  If you are not the intended recipient  or  if  you  have  received
this email in error, please delete it and  immediately  contact  the  sender  by
telephone or email, and destroy any copies  of  this  information.   You  should
not use or copy it, nor disclose  its  content  to  any  other  person  or  rely
upon this information.  Please note that any views presented in  the  email  and
any attachment hereto are solely those of the  author  and  do  not  necessarily
represent those of EURid.  While all care has been  taken  to  avoid  any  known
viruses, the recipient is advised to check this email  and  any  attachment  for
presence of viruses.

http://www.eurid.eu/en/legal-disclaimer