<div dir="ltr"><div>Thank you for your analysis, much appreciated. I've got more homework to do, to determine why breakage started, apparently spontaneously. We performed no service changes during the interval where the problem began to manifest.<br></div><div><br></div><div>Chad<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Nov 12, 2024 at 12:21 PM Viktor Dukhovni <<a href="mailto:ietf-dane@dukhovni.org">ietf-dane@dukhovni.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Tue, Nov 12, 2024 at 09:41:15AM -0600, Chad Dailey wrote:<br>
<br>
> Had an issue that started last week around 0304 CST 11/7/2024, mail<br>
> deliverability for our Exchange service went to zero. Looking closer it<br>
> appears that the DNSSEC chain is failing, because an invalid RCODE is being<br>
> returned.<br>
<br>
NOTIMP (rather than the correct NXDOMAIN or NODATA as appropriate)<br>
responses to queries for unexpected RRtypes by the nameservers for<br>
<a href="http://mail.protection.outlook.com" rel="noreferrer" target="_blank">mail.protection.outlook.com</a> is a well-known, longstanding behaviour (at<br>
least since Q1 2013, likely much earlier), but it is puzzling why your<br>
DNS server is running into the invalid responses.<br>
<br>
The zone in question is NOT signed, so non-EDNS fallback should happen<br>
automatically, and your queries should just be for A/AAAA records for<br>
which the nameservers return FORMERR as expected:<br>
<br>
$ dig +norecur +noall +comment +ans -t a <a href="http://nist-gov.mail.protection.outlook.com" rel="noreferrer" target="_blank">nist-gov.mail.protection.outlook.com</a> @<a href="http://ns1-proddns.glbdns.protection.outlook.com" rel="noreferrer" target="_blank">ns1-proddns.glbdns.protection.outlook.com</a>.<br>
;; Got answer:<br>
;; ->>HEADER<<- opcode: QUERY, status: FORMERR, id: 28750<br>
;; flags: qr; QUERY: 0, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0<br>
<br>
;; WARNING: EDNS query returned status FORMERR - retry with '+noedns'<br>
<br>
$ dig +noedns +norecur +noall +comment +ans -t a <a href="http://nist-gov.mail.protection.outlook.com" rel="noreferrer" target="_blank">nist-gov.mail.protection.outlook.com</a> @<a href="http://ns1-proddns.glbdns.protection.outlook.com" rel="noreferrer" target="_blank">ns1-proddns.glbdns.protection.outlook.com</a>.<br>
;; Got answer:<br>
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 35688<br>
;; flags: qr aa; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 0<br>
<br>
;; ANSWER SECTION:<br>
<a href="http://nist-gov.mail.protection.outlook.com" rel="noreferrer" target="_blank">nist-gov.mail.protection.outlook.com</a>. 10 IN A 52.101.8.50<br>
<a href="http://nist-gov.mail.protection.outlook.com" rel="noreferrer" target="_blank">nist-gov.mail.protection.outlook.com</a>. 10 IN A 52.101.9.19<br>
<a href="http://nist-gov.mail.protection.outlook.com" rel="noreferrer" target="_blank">nist-gov.mail.protection.outlook.com</a>. 10 IN A 52.101.8.52<br>
<a href="http://nist-gov.mail.protection.outlook.com" rel="noreferrer" target="_blank">nist-gov.mail.protection.outlook.com</a>. 10 IN A 52.101.11.12<br>
<br>
Generally speaking, there is not a pervasive problem with email to<br>
delivery to the domains they host, there is perhaps something unusual on<br>
your end that tickles the NOTIMP replies that others manage to avoid.<br>
<br>
> Forcing +noedns on our recursive boxes is our current<br>
> workaround, but would prefer not to leave this in place. Perhaps go chat<br>
> up your favorite DNS team member?<br>
> <br>
> <a href="https://dnsviz.net/d/mail.protection.outlook.com/dnssec/" rel="noreferrer" target="_blank">https://dnsviz.net/d/mail.protection.outlook.com/dnssec/</a><br>
> <br>
> Hit me up if you want the case number or other details.<br>
<br>
This is the main reason that RFC7672 includes a work-around (or if<br>
you like an optimisation) to skip TLSA queries for MX hosts whose<br>
address records are unsigned.<br>
<br>
<a href="https://datatracker.ietf.org/doc/html/rfc7672#section-2.2.2" rel="noreferrer" target="_blank">https://datatracker.ietf.org/doc/html/rfc7672#section-2.2.2</a><br>
<br>
Note that DNS queries with type TLSA are mishandled by load-balancing<br>
nameservers that serve the MX hostnames of some large email<br>
providers. The DNS zones served by these nameservers are not signed<br>
and contain no TLSA records. These nameservers SHOULD provide<br>
"insecure" negative replies that indicate the nonexistence of the<br>
TLSA records, but instead they fail by not responding at all or by<br>
responding with a DNS RCODE [RFC1035] other than NXDOMAIN, e.g.,<br>
SERVFAIL or NOTIMP [RFC2136].<br>
<br>
To avoid problems delivering mail to domains whose SMTP servers are<br>
served by these problematic nameservers, the SMTP client MUST perform<br>
any A and/or AAAA queries for the destination before attempting to<br>
locate the associated TLSA records. This lookup is needed in any<br>
case to determine (1) whether or not the destination domain is<br>
reachable and (2) the DNSSEC validation status of the chain of CNAME<br>
queries required to reach the ultimate address records.<br>
<br>
If no address records are found, the destination is unreachable. If<br>
address records are found but the DNSSEC validation status of the<br>
first query response is "insecure" (see Section 2.1.3), the SMTP<br>
client SHOULD NOT proceed to search for any associated TLSA records.<br>
...<br>
<br>
-- <br>
Viktor.<br>
_______________________________________________<br>
dns-operations mailing list<br>
<a href="mailto:dns-operations@lists.dns-oarc.net" target="_blank">dns-operations@lists.dns-oarc.net</a><br>
<a href="https://lists.dns-oarc.net/mailman/listinfo/dns-operations" rel="noreferrer" target="_blank">https://lists.dns-oarc.net/mailman/listinfo/dns-operations</a><br>
</blockquote></div>