[dns-operations] EDNS issue
Mark Andrews
marka at isc.org
Sat Feb 26 00:06:39 UTC 2011
In message <000201cbd546$9dd7be60$d9873b20$@iname.com>, "Frank Bulk" writes:
> # grep "reducing the advertised EDNS" /var/log/named.log | cut -b1-6 | uniq
> -c | more
> 14261 Feb 20
> 10218 Feb 21
> 8858 Feb 22
> 8916 Feb 23
> 9417 Feb 24
> 1559 Feb 25
>
> You can see that the count of those errors has really dropped since the
> change yesterday.
You will still get some messages logged even with a locally clean
EDNS path as it is timeouts that trigger the messages. They can
be triggered by:
* some unreachable servers for the zone.
* congestion.
* remote EDNS/DO failures.
* long rtt times (> 800 ms).
* other stuff.
If there was enough time it would be nice to be able to do this on
a per server basis rather than a per zone basis. Unfortunately
clients timeout too soon to do that.
> Frank
> -----Original Message-----
> From: dns-operations-bounces at lists.dns-oarc.net
> [mailto:dns-operations-bounces at lists.dns-oarc.net] On Behalf Of Frank Bulk
> Sent: Thursday, February 24, 2011 11:17 PM
> To: 'Mark Andrews'
> Cc: dns-operations at dns-oarc.net
> Subject: Re: [dns-operations] EDNS issue
>
> Cisco's documents recommend it, as well as did our consultant. =)
>
> Problems solved now that I removed all of them except the "icmp fragment" --
> thanks a million.
>
> Frank
>
> -----Original Message-----
> From: Mark Andrews [mailto:marka at isc.org]
> Sent: Thursday, February 24, 2011 11:01 PM
> To: frnkblk at iname.com
> Cc: dns-operations at dns-oarc.net
> Subject: Re: [dns-operations] EDNS issue
>
>
> In message <000b01cbd4a6$0740d4a0$15c27de0$@iname.com>, "Frank Bulk" writes:
> > Mark:
> >
> > Our borders routers blog fragments:
> >
> > Edge1#sh run | inc frag
> > access-list 101 deny tcp any any log fragments
> > access-list 101 deny udp any any log fragments
> > access-list 101 deny icmp any any log fragments
> > access-list 101 deny ip any any log fragments
> > Edge1#
> >
> > Not sure if that's the same kind of fragments as you're talking about.
> >
> > The first query failed, the second didn't. Do I need to be making some
> > changes to our border routers?
>
> Yes. You need to permit, as a minimum, fragmented UDP packets
> (determined by looking at the next header field of the IP header).
>
> There is this security myth that you can run a IP network without
> letting through fragmented IP packets. This has never been the
> case. There is also a security myth that fragmented IP packets are
> dangerous. They aren't. This is much the same as the security
> myth that ICMP packets are bad. ICMP is essential to the good
> running of a IP network.
>
> > root at nagios:/etc/cron.hourly# dig @140.172.17.237 +dnssec
> > forecast.weather.gov
> >
> > ; <<>> DiG 9.5.1-P3 <<>> @140.172.17.237 +dnssec forecast.weather.gov
> > ; (1 server found)
> > ;; global options: printcmd
> > ;; connection timed out; no servers could be reached
> > root at nagios:/etc/cron.hourly# dig @140.172.17.237 +dnssec
> > forecast.weather.gov +bufsize=1400
> >
> > ; <<>> DiG 9.5.1-P3 <<>> @140.172.17.237 +dnssec forecast.weather.gov
> > +bufsize=1400
> > ; (1 server found)
> > ;; global options: printcmd
> > ;; Got answer:
> > ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 12832
> > ;; flags: qr aa rd; QUERY: 1, ANSWER: 8, AUTHORITY: 4, ADDITIONAL: 4
> > ;; WARNING: recursion requested but not available
> >
> > ;; OPT PSEUDOSECTION:
> > ; EDNS: version: 0, flags: do; udp: 4096
> > ;; QUESTION SECTION:
> > ;forecast.weather.gov. IN A
> >
> > ;; ANSWER SECTION:
> > forecast.weather.gov. 5 IN CNAME edge-ext.lb.noaa.gov.
> > forecast.weather.gov. 5 IN RRSIG CNAME 5 3 5 20110303213142
> > 20110224213142 17808 weather.gov.
> > B/ga+/E+QUWvvGxAbCqcekDsZj0H0cCnYtB3n4/SPiRaGQZyLTbKHAFr
> > U2wsBQdecrAwtcbMA1eIHpqVw1c15H6L6m2931fUJ0uUXk9ahBiBIf5h
> > ztst05qoFrbotqFiZjO4sYTOK02YxSnmSWv0RpFIYzMGd9vYL4mpC7EF
> > mM0/97QBJTMI51dZPU2tLuLXYc7ZX3psKM145r90vp3HMZU4h+dCkl6S
> > QcVq23qy/3UHvw50G1N3HHwIp0asExGQ2dJilWDk/1/jsAAV87GzA+sg
> > +0pjPnd45Rdd/HC1thj/Bq4KK6FOLzzWFAey5HpWvxt1o5Mnh7c8rPeK V8puqA==
> > edge-ext.lb.noaa.gov. 30 IN A 129.15.96.22
> > edge-ext.lb.noaa.gov. 30 IN A 140.90.33.21
> > edge-ext.lb.noaa.gov. 30 IN A 140.90.33.22
> > edge-ext.lb.noaa.gov. 30 IN A 140.90.200.22
> > edge-ext.lb.noaa.gov. 30 IN A 140.172.17.22
> > edge-ext.lb.noaa.gov. 30 IN RRSIG A 5 4 30 20110304033622
> > 20110225033622 38565 lb.noaa.gov.
> > qAGLlVt9kZbwvP9DGodd+a/IHX/l+jY/7Pkk7Pcss4nuaKEL01H5TmsF
> > 37CPDz8oaSHSX0PnVz7aMh3LDAjyEupgF7Rcqh0eu+9bQQBy8D+/h4LI
> > 7Zrqa+t2Km4Ummx6Xn3miHqHHFyheMlMvUKXzdQZRvwrMYYkAPHYDFU/
> > RB23/jRb5ng5tHvyuFz2rp5TjL4j1wJbusHKkwa9EHJPEjmPPTufNz6/
> > aVM69UN/sIjwhr/JMjonwLswk/5+PUu4FieOl6ot6d/8HPn9/x5uad+N
> > uKY9v0RV7pDvqnUyR0rR+vWCHu5TGVc/8MKNRmdToNmGQNEFXGxxn1sB Fh6Djg==
> >
> > ;; AUTHORITY SECTION:
> > lb.noaa.gov. 86400 IN NS ns-mw.noaa.gov.
> > lb.noaa.gov. 86400 IN NS ns-nw.noaa.gov.
> > lb.noaa.gov. 86400 IN NS ns-e.noaa.gov.
> > lb.noaa.gov. 86400 IN RRSIG NS 5 3 86400
> 20110304033622
> > 20110225033622 38565 lb.noaa.gov.
> > MWFCzQO4u74tqr7lxAuzT5LEEZPl45BHabC5ftC96Ufd8GB7n/AqOppT
> > kO01bwAZBt30FBGShq1R+wc0nPFlLzJE5flyvA5dJwlF6jDh5fL9xt80
> > UFEwlGdS/ogPeKdgNKrIMQ+VlMn5sd3rhaff9+rfIH3Bx38B8pZiQt5q
> > Ii+vll/ASbHiLO91G6b6Ht8XoWn2y/jwQVAhApCI8DMswISqUZcQVoix
> > V9OllJowlvuEJx/lFZYHjMibnHEZsSLr+mccMvB0tt46fdm3u7aDsUHr
> > OrTLG6bh60AnyYtb+zExc2odyzlEx+AE+U72HWn7fhHg7HRbiXzYvfcZ x5yrOw==
> >
> > ;; ADDITIONAL SECTION:
> > ns-e.noaa.gov. 86400 IN A 140.90.33.237
> > ns-mw.noaa.gov. 86400 IN A 140.172.17.237
> > ns-nw.noaa.gov. 86400 IN A 161.55.32.2
> >
> > ;; Query time: 55 msec
> > ;; SERVER: 140.172.17.237#53(140.172.17.237)
> > ;; WHEN: Thu Feb 24 22:36:56 2011
> > ;; MSG SIZE rcvd: 1164
> >
> > root at nagios:/etc/cron.hourly#
> >
> > -----Original Message-----
> > From: Mark Andrews [mailto:marka at isc.org]
> > Sent: Thursday, February 24, 2011 7:27 PM
> > To: frnkblk at iname.com
> > Cc: dns-operations at dns-oarc.net
> > Subject: Re: [dns-operations] EDNS issue
> >
> >
> > In message <006d01cbd482$d54e5a30$7feb0e90$@iname.com>, "Frank Bulk"
> writes:
> > > Our ISP helpdesk has been receiving a lot of complaints about their
> > > inability to check the weather weather.gov, specifically,
> > > forecast.weather.gov. Some digs showed that queries were failing, and
> my
> > > BIND logs show the same:
> >
> > Make sure you can receive fragmented UDP responses. The servers
> > are sending good reponses.
> >
> > ;; Query time: 201 msec
> > ;; SERVER: 140.172.17.237#53(ns-mw.noaa.gov)
> > ;; WHEN: Fri Feb 25 12:17:01 2011
> > ;; MSG SIZE rcvd: 2052
> >
> > Try the following two queries. The first response will be fragmented
> > and the second shouldn't be fragmented.
> >
> > dig @140.172.17.237 +dnssec forecast.weather.gov
> > dig @140.172.17.237 +dnssec forecast.weather.gov +bufsize=1400
> >
> > Mark
> >
> > > Feb 24 18:25:12 10.20.0.100 named[2603]: too many timeouts resolving
> > > 'forecast.weather.gov/A' (in 'weather.gov'?): disabling EDNS
> > > Feb 24 18:25:36 199.120.69.22 named[5289]: success resolving
> > > 'forecast.weather.gov/A' (in 'weather.gov'?) after reducing the
> advertised
> > > EDNS UDP packet size to 512 octets
> > > Feb 24 18:25:37 10.20.0.200 named[2583]: success resolving
> > > 'forecast.weather.gov/A' (in 'weather.gov'?) after reducing the
> advertised
> > > EDNS UDP packet size to 512 octets
> > > Feb 24 18:25:38 10.20.0.100 named[2603]: too many timeouts resolving
> > > 'forecast.weather.gov/A' (in 'weather.gov'?): disabling EDNS
> > > Feb 24 18:25:39 199.120.69.22 named[5289]: success resolving
> > > 'radar.weather.gov/A' (in 'weather.gov'?) after reducing the advertised
> > EDNS
> > > UDP packet size to 512 octets
> > > Feb 24 18:25:40 199.120.69.22 named[5289]: success resolving
> > > 'www.weather.gov/A' (in 'weather.gov'?) after reducing the advertised
> EDNS
> > > UDP packet size to 512 octets
> > > Feb 24 18:25:42 10.20.0.200 named[2583]: success resolving
> > > 'radar.weather.gov/A' (in 'weather.gov'?) after reducing the advertised
> > EDNS
> > > UDP packet size to 512 octets
> > > Feb 24 18:25:42 10.20.0.100 named[2603]: too many timeouts resolving
> > > 'radar.weather.gov/A' (in 'weather.gov'?): disabling EDNS
> > >
> > >
> > > A quick check showed the following:
> > >
> > > mail1:~# dig -4 +short rs.dns-oarc.net txt
> > > rst.x1002.rs.dns-oarc.net.
> > > rst.x1994.x1002.rs.dns-oarc.net.
> > > rst.x2495.x1994.x1002.rs.dns-oarc.net.
> > > "Tested at 2011-02-25 00:20:31 UTC"
> > > "2607:fe28:0:1003:223:7dff:fe9c:4aa5 sent EDNS buffer size 4096"
> > > "2607:fe28:0:1003:223:7dff:fe9c:4aa5 DNS reply size limit is at
> > > least 2495"
> > > mail1:~#
> > > mail1:~# dig forecast.weather.gov
> > >
> > > ; <<>> DiG 9.3.4-P1.1 <<>> forecast.weather.gov
> > > ;; global options: printcmd
> > > ;; connection timed out; no servers could be reached
> > >
> > > Does this make sense? EDNS of size 512 shouldn't be an issue, yet all 7
> > > *nix DNS servers (the first one is above) running BIND complain. The
> > first
> > > four of the DNS servers are behind an old F5 BigIP, the others aren't.
> > >
> > > root at nagios:/var/log# dig -4 +short rs.dns-oarc.net txt
> > > rst.x1002.rs.dns-oarc.net.
> > > rst.x1222.x1002.rs.dns-oarc.net.
> > > rst.x1403.x1222.x1002.rs.dns-oarc.net.
> > > "96.31.0.5 DNS reply size limit is at least 1403"
> > > "Tested at 2011-02-25 00:24:38 UTC"
> > > "96.31.0.5 sent EDNS buffer size 4096"
> > > root at nagios:/var/log#
> > > root at nagios:/var/log# dig forecast.weather.gov
> > >
> > > ; <<>> DiG 9.5.1-P3 <<>> forecast.weather.gov
> > > ;; global options: printcmd
> > > ;; connection timed out; no servers could be reached
> > > root at nagios:/var/log#
> > >
> > > Any ideas? Querying our corporate Microsoft DNS server, behind a Cisco
> > ASA,
> > > works fine!
> > >
> > > Frank Bulk
> > >
> > > _______________________________________________
> > > dns-operations mailing list
> > > dns-operations at lists.dns-oarc.net
> > > https://lists.dns-oarc.net/mailman/listinfo/dns-operations
> > --
> > Mark Andrews, ISC
> > 1 Seymour St., Dundas Valley, NSW 2117, Australia
> > PHONE: +61 2 9871 4742 INTERNET: marka at isc.org
> >
> --
> Mark Andrews, ISC
> 1 Seymour St., Dundas Valley, NSW 2117, Australia
> PHONE: +61 2 9871 4742 INTERNET: marka at isc.org
>
> _______________________________________________
> dns-operations mailing list
> dns-operations at lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations
>
--
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742 INTERNET: marka at isc.org
More information about the dns-operations
mailing list