<div dir="ltr"><div>Hi,</div><div><br></div><div>sorry to dredge this back up, but I just want to give anyone the chance to object. <br></div><div><br></div><div>My read of what Viktor and others have indicated here is that, when a validating resolver receives a response with expired rrsigs, it's okay (and encouraged?) for that resolver to treat that as an invalid response and retry against other nameservers, similarly to how it would handle a REFUSED or SERVFAIL response from an authority (i.e. with similar care to limit retry storms). <br></div><div><br></div><div>The purpose of this is so that a single stale pop or authoritative host would not cause an outage to dnssec signed domains, as resolvers will retry against others.</div><div><br></div><div>I'd like to reach out to NLNet about changing Unbound to do this, so I want to make sure people have a chance to disagree. Feel free to voice your disagreement (and reasons) here if you do.<br></div><div><br></div><div>Gavin </div><div><br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jul 13, 2023 at 2:21 PM Gavin McCullagh <<a href="mailto:gmccullagh@gmail.com">gmccullagh@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hi,<br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jul 13, 2023 at 1:18 PM Viktor Dukhovni <<a href="mailto:ietf-dane@dukhovni.org" target="_blank">ietf-dane@dukhovni.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Thu, Jul 13, 2023 at 12:16:37PM -0700, Gavin McCullagh wrote:<br>
<br>
> When faced with ~4x obviously bogus, broken nameservers (the stale pop) and<br>
> ~9x fresh working nameservers with valid signatures, the DNSSEC RFCs appear<br>
> to specify (and Unbound appears to implement) that resolvers must accept<br>
> and cache the obviously bogus (expired rrsig) answers and return SERVFAIL<br>
> to clients until those bogus answers expire (apparently 24 hours later in<br>
> this case?), rather than immediately considering those responses invalid,<br>
> those nameservers as lame and retrying against another - assuming this<br>
> response is either coming from an attacker or from a broken nameserver.<br>
<br>
I don't believe that's a valid reading of the DNSSEC RFCs. Bogus<br>
answers are not cached by the resolver I'm working on these days, beyond<br>
the usual query rate limits for failures.<br>
<br>
And to the extend that a only some servers have stale data, I am also<br>
working to make sure that we'll try go get a better answer from another<br>
server. Some resolvers already do.<br></blockquote><div><br></div><div>Good to hear, thanks. The text seems open to the interpretation that a response with bad RRSIGs should result in RCODE 2 <a href="https://datatracker.ietf.org/doc/html/rfc4035#section-5.5" target="_blank">https://datatracker.ietf.org/doc/html/rfc4035#section-5.5</a> :<br></div><div><pre> If for whatever reason none of the RRSIGs can be validated, the
response SHOULD be considered BAD. If the validation was being done
to service a recursive query, the name server MUST return RCODE 2 to
the originating client. However, it MUST return the full response if
and only if the original query had the CD bit set. Also see <a href="https://datatracker.ietf.org/doc/html/rfc4035#section-4.7" target="_blank">Section</a>
<a href="https://datatracker.ietf.org/doc/html/rfc4035#section-4.7" target="_blank">4.7</a> on caching responses that do not validate.</pre></div><div><br></div><div>Do you think it's worth clarifying? Or maybe I am just looking at the wrong RFC?<br></div><div><br></div><div>The impact, seemingly to unbound and likely at least one other implementation, seems to suggest that either a) those resolvers were not retrying or b) they were retrying but not often enough to reliably get past 4/13 stale nameservers. The logs indicate this was being cached too. The only mitigation was basically to disable validation and restart.<br></div><div><br></div><div><pre style="box-sizing:inherit;margin:4px 0px;padding:8px;font-size:12px;font-variant-ligatures:none;line-height:1.50001;white-space:pre-wrap;word-break:normal;font-family:Monaco,Menlo,Consolas,"Courier New",monospace;border-radius:4px;overflow-y:hidden;color:rgb(29,28,29);font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:left;text-indent:0px;text-transform:none;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial"> Jul 10 07:59:48 host2 unbound: [36427:1] info: validation failure <<a href="http://domain.in.question.xxxx.com" target="_blank">domain.in.question.xxxx.com</a> A IN>: key for validation xxxx<a href="http://amazonaws.com/" rel="noopener noreferrer" style="box-sizing:inherit;color:inherit;text-decoration:none" target="_blank">.com</a>. is marked as invalid because of a previous validation failure <<a href="http://some.other.xxxx.com" target="_blank">some.other.xxxx.com</a>. A IN>: signature expired from 192.26.92.30 for DS xxxx<a href="http://amazonaws.com/" rel="noopener noreferrer" style="box-sizing:inherit;color:inherit;text-decoration:none" target="_blank">.com</a>. while building chain of trust</pre></div><div><br></div><div>I suspect the fact the <span>impact </span>was mild, was primarily because many do not enable DNSSEC validation in critical places today. At least one big provider who had validation enabled had to re-route away from the bad PoP (awesome they could do it, but that's not a reasonable expectation of everyone). Had validation been widely enabled, I suspect this would have been a significant event. <br></div><div><br></div><div>We've experienced a stale anycast pop over the years. It's not an easy problem to completely avoid, so I'm sympathetic and think we should try to be resilient to it.<br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Within reasonable retry limit counts and error response<br>
hold-downs, bad/stale/... data from a single server should not be final<br>
whether it is DNSSEC-related or not.<br></blockquote><div><br></div><div>I see. We had guessed that the stale answer was being treated as valid, cacheable authoritative data, similar to how an NXDOMAIN would be - and a resolver would not retry. If that's not the case, that's reassuring, but it would be great to make sure the rfcs and implementations all agree on it.<br></div><div><br></div><div>Gavin<br></div><div> </div></div></div>
</blockquote></div>