[dns-operations] service-now.com DNSSEC broken?
michael at brokendns.net
Mon Jan 30 05:29:45 UTC 2017
On 01/27/17 19:21, Casey Deccio wrote:
>> On Jan 27, 2017, at 8:03 PM, David <opendak at shaw.ca> wrote:
>> On 2017-01-27 5:33 PM, Stephan Lagerholm wrote:
>>> 188.8.131.52 as well as Comcast DNS is servfailing without the +cd bit. But
>>> domain comes out clean in the verisign and dnsviz debuggers.
>>> Can anybody help me figure out what is wrong with it?
> Looks like it is intermittent:
> http://dnsviz.net/d/service-now.com/e/126998240/dnssec/ (looks good)
> http://dnsviz.net/d/service-now.com/e/127004107/dnssec/ (looks broken)
> (Both examples query 184.108.40.206)
> Perhaps the DS/DNSKEY inconsistency exists in some of the backend caches of Google and Comcast, but not everywhere. Even when the correct records (i.e., matching DS/DNSKEY) are returned by 220.127.116.11, SERVFAIL is returned without the CD bit set in the request .
In the cases I queried, which led to my post to outages@, 18.104.22.168
responded with different DS keytags to repeated queries from the same
source host. My conclusion is that some Google backend caches had
expired the old DS record (keytag==30126) and some had not (as is to be
expected). Because repeated queries to 22.214.171.124 from the same source
host create different cache responses, my assumption is that dnsviz
can't guarantee a response from the same backend cache on two successive
queries. So, even if the DS/DNSKEY responses looked the same, I don't
think we can know that they actually came from the same backend cache.
Doing cache dumps on our internal BIND 9.11 recursive server that was
giving SERVFAILs shows the cached DS record with keytag of 30126, while
the only KSK in the DNSKEY had tag 31893. The TTL on the DS record in
com is 86400 and the TTL on the DNSKEY record was 7200. This feels like
a simple failure to respect the DS record TTL--i.e. a botched KSK roll.
"Interestingly,"** it appears that there have been two recent KSK rolls
for service-now.com. For many months the primary (i.e. referenced in
the parent DS and signing the ZSKs) KSK had tag 42762, with 30126 being
published but not signing anything. During the month of January, 30126
was rolled in as primary around 1/13/17, only to be hastily (and
improperly) rolled again 2 weeks later (making 31983 the primary). Not
sure if something went wrong with 30126 or if a 2-week KSK lifetime was
** I put "interestingly" in quotes because a friend of mine told me
several years ago that whenever a tech person puts "interestingly" in a
sentence, what follows is rarely actually interesting to normal people.
More information about the dns-operations