[dns-operations] in-addr.arpa spikes in DNS traffic

Wed Jan 9 17:10:24 UTC 2019

Hypothesis:
Large network scans or ddos attacks against cloud provider could cause that. Firewalls and also "personal" firewalls of consumer  broadband connections tries to resolve the PTR RRs for logging or analysis purposes.

Winfried

Am 9. Januar 2019 15:40:28 MEZ schrieb "MONROE, JEREMY" <jm9386 at att.com>:
>To clarify – We are seeing the spikes in PTR/in-addr.arpa lookups on
>domains we are authoritative for in the ISP arena – recurring on 16
>hour intervals.  I’ve received reports that at least one other ISP has
>noticed the same thing.
>
>Jeremy Monroe
>Q Me qto://talk/jm9386
>Senior - IT Network Design
>Enterprise IP Services Support
>AT&T Services Inc – Network Cloud and Infrastructure Ops
>Intranet: http://eiss.it.att.com<http://eiss.it.att.com/>
>          http://eiss-dns.it.att.com<http://eiss-dns.it.att.com/>
>573 204 5463 Skype/office
>314 235 8168 (AT&T Domains voice mailbox)
>3146508345 at txt.att.net<mailto:3146508345 at txt.att.net> Pager
>314 650 8345 Cell
>"I don't know the secret to success, but the secret to failure is to
>try and please everyone"
>
>"This e-mail and any files transmitted with it are the property of
>at&t, are confidential, and are intended solely for the use of the
>individual or entity to whom this e-mail is addressed. If you are not
>one of the named recipient or otherwise have reason to believe that you
>have received this message in error, please notify the sender at
>[jm9386 at att.com or 314 235 8168] and delete this message immediately
>from your computer. Any other use, retention, dissemination,
>forwarding, printing, or copying of this e-mail is strictly
>prohibited."
>
>From: Hellqvist, Björn <bjorn.hellqvist at teliacompany.com>
>Sent: Monday, January 07, 2019 02:40
>To: MONROE, JEREMY <jm9386 at att.com>; dns-operations at dns-oarc.net;
>ask-rssac at icann.org
>Subject: RE: in-addr.arpa spikes in DNS traffic
>
>
>Hi,
>
>We have also noticed similar bursts in the past. Maybe not at those
>rates, but significantly larger than normal. In the magnitude of 10-20x
>of normal traffic, and only from a small amount of sources.
>
>The main IP’s at the time was AWS servers. My conclusion at the time
>was that some sites did reverse lookups without using a caching
>resolver. I might have reached out to Amazon, but I can’t remember. It
>was a couple of years ago.
>
>I also saw some of the larger Swedish web hosting companies at that
>time, which I did reach out to.
>
>Since the traffic was no threat of overloading our authoritative
>servers, I let them pass.
>
>Have not looked at it for a while though.
>
>BR,
>Bjorn Hellqvist
>Senior System Expert (Internet, DNS & Automation)
>Telia Company
>Solna, Sweden
>
>
>
>From: dns-operations [mailto:dns-operations-bounces at dns-oarc.net] On
>Behalf Of MONROE, JEREMY
>Sent: den 22 december 2018 16:35
>To: dns-operations at dns-oarc.net<mailto:dns-operations at dns-oarc.net>;
>ask-rssac at icann.org<mailto:ask-rssac at icann.org>
>Subject: [dns-operations] in-addr.arpa spikes in DNS traffic
>
>Please see below – AT&T has received bursts in PTR queries as described
>below – has anyone hear seen similar behavior recently?  This first
>occurred in March of 2018 – subsided and began again here in December.
>
>Jeremy Monroe
>Q Me qto://talk/jm9386
>Senior - IT Network Design
>Enterprise IP Services Support
>AT&T Services Inc – Network Cloud and Infrastructure Ops
>Intranet: http://eiss.it.att.com<http://eiss.it.att.com/>
>          http://eiss-dns.it.att.com<http://eiss-dns.it.att.com/>
>573 204 5463 Skype/office
>314 235 8168 (AT&T Domains voice mailbox)
>3146508345 at txt.att.net<mailto:3146508345 at txt.att.net> Pager
>314 650 8345 Cell
>"I don't know the secret to success, but the secret to failure is to
>try and please everyone"
>
>"This e-mail and any files transmitted with it are the property of
>at&t, are confidential, and are intended solely for the use of the
>individual or entity to whom this e-mail is addressed. If you are not
>one of the named recipient or otherwise have reason to believe that you
>have received this message in error, please notify the sender at
>[jm9386 at att.com or 314 235 8168] and delete this message immediately
>from your computer. Any other use, retention, dissemination,
>forwarding, printing, or copying of this e-mail is strictly
>prohibited."
>
>From: Wessels, Duane
><dwessels at verisign.com<mailto:dwessels at verisign.com>>
>Sent: Friday, December 21, 2018 16:05
>To: MONROE, JEREMY <jm9386 at att.com<mailto:jm9386 at att.com>>
>Subject: Re: RE: in-addr.arpa spikes in DNS traffic
>
>Yeah that’s an odd one.  If the IPs are real (not spoofed) and you see
>queries that you are authoritative for, then it would point to
>something generating large amounts of queries through legitimate
>recursives.
>
>We asked our root server colleagues they saw anything like that.  Only
>one responded so far, and said (like us) they did not.
>
>As you may know, root servers typically handle in the range of 50k q/s
>these days.  The 200-400k that you observe would definitely be
>noticeable.
>
>Currently the only reasonable way you could reach all the root
>operators is to send email to
>ask-rssac at icann.org<mailto:ask-rssac at icann.org>.  Someone would receive
>it and then forward it to the operators.  Given the timing with
>holidays I wouldn’t hold your breath.
>
>You might also consider posting to the
>dns-operations at dns-oarc.net<mailto:dns-operations at dns-oarc.net> mailing
>list to reach other DNS operators more broadly.
>
>DW
>
>
>From: "MONROE, JEREMY" <jm9386 at att.com<mailto:jm9386 at att.com>>
>Date: Friday, December 21, 2018 at 1:48 PM
>To: Duane Wessels <dwessels at verisign.com<mailto:dwessels at verisign.com>>
>Subject: [EXTERNAL] RE: in-addr.arpa spikes in DNS traffic
>
>We are seeing short bursts of PTR query traffic.  Sources seem to be
>all open ISP resolvers scattered all over the United States.  I host a
>bunch of in-addr.arpa zones from top level delegations and each of the
>queries appears to be for legitimate PTR records that we have defined. 
>We typically receive about 7-10kpps (Packets Per Second) (not terribly
>large) and over the last week or two have received what seems like
>coordinated bursts of up and over 200-400kpps of all PTR records. 
>First observed in the United States – but today I learned our European
>based resolvers have also received similar spikes.  My assumption was
>that if a bunch of ISP resolvers began receiving PTR queries for
>recursion – that the root servers might have seen an increase in folks
>asking what DNS servers to use for certain in-addr.arpa space at AT&T. 
>We have not seen a significant number of queries for arpa’s that we are
>not authoritative for – it’s an odd MO.
>
>Thank you for taking the time to reply back.  Is there any way to see
>if any of the other root-server providers have noticed anything of that
>sort?  Im really grasping at straws at this point.
>
>Jeremy Monroe
>Q Me qto://talk/jm9386
>Senior - IT Network Design
>Enterprise IP Services Support
>AT&T Services Inc – Network Cloud and Infrastructure Ops
>Intranet: http://eiss.it.att.com<http://eiss.it.att.com/>
>          http://eiss-dns.it.att.com<http://eiss-dns.it.att.com/>
>573 204 5463 Skype/office
>314 235 8168 (AT&T Domains voice mailbox)
>3146508345 at txt.att.net<mailto:3146508345 at txt.att.net> Pager
>314 650 8345 Cell
>"I don't know the secret to success, but the secret to failure is to
>try and please everyone"
>
>"This e-mail and any files transmitted with it are the property of
>at&t, are confidential, and are intended solely for the use of the
>individual or entity to whom this e-mail is addressed. If you are not
>one of the named recipient or otherwise have reason to believe that you
>have received this message in error, please notify the sender at
>[jm9386 at att.com or 314 235 8168] and delete this message immediately
>from your computer. Any other use, retention, dissemination,
>forwarding, printing, or copying of this e-mail is strictly
>prohibited."
>
>From: Wessels, Duane
><dwessels at verisign.com<mailto:dwessels at verisign.com>>
>Sent: Friday, December 21, 2018 15:37
>To: MONROE, JEREMY <jm9386 at att.com<mailto:jm9386 at att.com>>
>Subject: Re: in-addr.arpa spikes in DNS traffic
>
>Jeremy,
>
>We don’t see anything like that here.  You mentioned both PTR and NS
>queries.  Are you seeing both, or is it one or the other?
>
>What you describe could be caused by availability issues with the lower
>levels of the DNS.  Did you notice any similarities in the names being
>queried?
>
>DW
>
>
>From: "MONROE, JEREMY" <jm9386 at att.com<mailto:jm9386 at att.com>>
>Date: Friday, December 21, 2018 at 7:52 AM
>To: rootdns <rootdns at verisign.com<mailto:rootdns at verisign.com>>
>Subject: [EXTERNAL] in-addr.arpa spikes in DNS traffic
>
>Hello – Im looking into a few network events where we received huge
>spikes in what appears to be valid PTR record lookups for zones to
>which we are authoritative for.  Can you confirm whether or not the
>root servers have seen similar spikes in in-addr.arpa related NS
>queries?
>
>Jeremy Monroe
>Q Me qto://talk/jm9386
>Senior - IT Network Design
>Enterprise IP Services Support
>AT&T Services Inc – Network Cloud and Infrastructure Ops
>Intranet: http://eiss.it.att.com<http://eiss.it.att.com/>
>          http://eiss-dns.it.att.com<http://eiss-dns.it.att.com/>
>573 204 5463 Skype/office
>314 235 8168 (AT&T Domains voice mailbox)
>3146508345 at txt.att.net<mailto:3146508345 at txt.att.net> Pager
>314 650 8345 Cell
>"I don't know the secret to success, but the secret to failure is to
>try and please everyone"
>
>"This e-mail and any files transmitted with it are the property of
>at&t, are confidential, and are intended solely for the use of the
>individual or entity to whom this e-mail is addressed. If you are not
>one of the named recipient or otherwise have reason to believe that you
>have received this message in error, please notify the sender at
>[jm9386 at att.com or 314 235 8168] and delete this message immediately
>from your computer. Any other use, retention, dissemination,
>forwarding, printing, or copying of this e-mail is strictly
>prohibited."
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.dns-oarc.net/pipermail/dns-operations/attachments/20190109/509cd7e5/attachment-0001.html>