<div id="geary-body" dir="auto"><div><br></div></div><div id="geary-quote" dir="auto"><br>On Tue, Oct 27, 2020 at 12:24 AM, Viktor Dukhovni <ietf-dane@dukhovni.org> wrote:<br><blockquote type="cite"><div class="plaintext" style="white-space: pre-wrap;">On Mon, Oct 26, 2020 at 10:08:31PM -0400, Viktor Dukhovni wrote:
<blockquote> qname | flags | alg | stime | etime | kid
---------------+-------+-----+------------+------------+------------
agrilinks.org | 257 | 8 | 2018-07-29 | 2020-07-22 | 26428972
agrilinks.org | 257 | 13 | 2020-07-22 | | 486
[...]
agrilinks.org | 256 | 8 | 2020-07-11 | 2020-07-22 | 4024196652
agrilinks.org | 256 | 13 | 2020-07-22 | | 29460111
The rather low KSK (key row id) ("kid") of 486, is due to the fact that
the P256 key in question has been in use as a KSK over the last ~3 years
by ~262,991 distinct domains and is still in use by ~206,413 of them.
</blockquote>
I might note that my "kid" 29460111 is also shared by many domains
(presently at least 251,300) and has been in use for over two years.</div></blockquote><div style="white-space: pre-wrap;"><span style="white-space: pre-wrap;"><br></span></div><span style="white-space: pre-wrap;">I'd expect Cloudflare to have more than 206k customers, so clearly not their sole signing key. 200k to 300k zones per signing server sounds plausible, each (as you say below) with its own HSM.</span><div><span style="white-space: pre-wrap;"><br></span><blockquote type="cite"><div class="plaintext" style="white-space: pre-wrap;">So it appears that domains using this KSK and ZSK (likely stored in
hardened HSMs, ...) tend to keep using them long-term. It looks
therefore like your domain was migrated to Cloudflare back in July,
but lack of ZSK rollovers since is not immediately indicative of any
latent issue.</div></blockquote><span style="white-space: pre-wrap;"><div><span style="white-space: pre-wrap;"><br></span></div><div><span style="white-space: pre-wrap;">Sort of. It was migrated from one CF account to another. Doesn't "explain" why the DNSKEY's RRSIG was allowed to expire, but may narrow down the possible code paths. When it was in the prior account, it was likely on older hardware with an RSA(8) ZSK and then moved as part of the transfer. We had to update the nameservers in WHOIS as part of the migration from one set of CF name servers to another.</span></div><div><span style="white-space: pre-wrap;"><br></span></div><div><span style="white-space: pre-wrap;">I'm less concerned about the lack of a ZSK rollover per se, and more that the RRSIG covering it wasn't regenerated in a timely manner. All we can really say is it wasn't caused by a rollover, but if the RRSIG and the rollover are always done at the same time, that would explain the expiring RRSIG. As you p
oint out, the past history of monthly ZSK rollovers on the prior account, and none in the three months for the current account -- is that a bug in their zone management or is it just a different policy for ECDSA(13) vs RSA(8) ZSKs?</span></div><div><span style="white-space: pre-wrap;"><br></span></div><div><span style="white-space: pre-wrap;">Put another way, is it one bug or two bugs, one that prevented a rollover and one that prevented regenerating the RRSIG? And, is it worthwhile to keep the latter bug as a canary?</span></div><br></span><blockquote type="cite"><div class="plaintext" style="white-space: pre-wrap;">What's more, it seems to now have a valid signature, so it looks like
you reached the right folks via some channel or other.
agrilinks.org. IN DNSKEY 257 3 13 mdsswUyr3DPW132mOi8V9xESWE8jTo0dxCjjnopKl+GqJxpVXckHAeF+KkxLbxILfDLUT0rAK9iUzy1L53eKGQ==
agrilinks.org. IN DNSKEY 256 3 13 oJMRESz5E4gYzS/q6XDrvU1qMPYIjCWzJaOau8XNEZeqCYKD5ar0IRd8KqXXFJkqmVfRvMGPmM1x8fGAa2XhSA==
agrilinks.org. IN RRSIG DNSKEY 13 2 3600 20201126034731 20200927034731 2371 agrilinks.org. yjY9OSOLtMViN8ZYL/J0uaUGzTtJcHoyzP5WhMXIXqqF99YONh4AkmL0D1kOBkWKFnwqseU8vFbME8BmigQRxA==<br></div></blockquote><div style="white-space: pre-wrap;"><span style="white-space: pre-wrap;"><br></span></div><span style="white-space: pre-wrap;">Yes, toggling DNSSEC off and on in the web dashboard forced a re-signing of the zone. I don't consider that to be more than a quick-fix for the issue, as I don't want to have to go into the dash every month or so to click a button, wait, and click it again.</span><br><br><span style="white-space: pre-wrap;">Thank you for your advice and the zone history. I have some more pointed questions for them now and data to support the questions.</span></div><div><span style="white-space: pre-wrap;"><br></span></div><div><span style="white-space: pre-wrap;">I'm going to recommend to our team that we add some DNS monitoring to track the DNSKEY RRSIG expiration dates.
Other RRs in the zone get 2-day (50 hour) RRSIGs, with an inception date roughly one day in the past. There simply aren't 3.14 days for a warning track, making pre-failure monitoring harder and more likely to generate critical alerts outside normal business hours. </span></div><div><span style="white-space: pre-wrap;"><br></span><div><div><span style="white-space: pre-wrap;">jf</span></div><div><span style="white-space: pre-wrap;">-- </span></div><div><span style="white-space: pre-wrap;">John Franklin</span></div><div><span style="white-space: pre-wrap;">franklin@sentaidigital.com</span></div></div></div></div>