[dns-operations] specifics of UDP response with truncate bit; odd google fail on AAAA responses w/ truncation

Fri May 25 02:02:08 UTC 2012

Roy-

How are you attempting to replicate?  I have been able to against the
Google resolver and directly against AWS.  IPv4/IPv6 won't matter for the
actual query.

-Dave

On 5/24/12 9:55 PM, "Roy Rapoport" <rrapoport at netflix.com> wrote:

>Answers to your questions below, Ryan, based on my understanding of DNS
>and my updated reading of RFC1035 (there does not seem to be a newer RFC
>which obsoletes 1035's stance on the issues below):
>
>- Should a packet with the truncate bit set contain answers, or is this
>optional?  I'm guessing optional, but could see arguments for the UDP
>response with the truncate bit containing at least the first few RRs
>
>RFC 2181 (http://www.zoneedit.com/doc/rfc/rfc2181.txt) states, on page 10,
>section 9:
>
>"Where TC is set, the partial RRSet that would not completely fit may
>   be left in the response.  When a DNS client receives a reply with TC
>   set, it should ignore that response, and query again, using a
>   mechanism, such as a TCP connection, that will permit larger replies"
>
>Notice the use of the word "may" in the first line above.  There does not
>appear to be a requirement that a truncated response contain an actual
>ANSWER section with RRs in it.
>
>
>- Should a packet with the truncate bit set have the field for the number
>of Answers reflect how many answers are in that packet, or how many are in
>the actual forthcoming response?  I believe that it should contain the
>number of RRs contained in the UDP response itself, not the full answer to
>the query - and this is where I believe the Amazon response is malformed.
>In the UDP response it says there are 24 answer RRs when there are zero
>
>RFC 1035 (http://www.ietf.org/rfc/rfc1035.txt) states, on page 27, 4.1.1:
>
>
>"ANCOUNT         an unsigned 16 bit integer specifying the number of
>                resource records in the answer section."
>
>This appears to be a very clear answer to your question: The ANCOUNT
>should represent the number of records RETURNED, not the records available
>if you were to get an untruncated response.
>
>Right now, I've not yet been able to replicate the error you see, BTW, but
>I suspect this is due to being on an IPv4 network.
>
>-roy
>
>
>
>
>
>
>On 5/24/12 6:38 PM, "Ryan Rawdon" <ryan at u13.net> wrote:
>
>>Since Netflix added AAAAs to movies.netflix.com (or more specifically,
>>enabled IPv6 on the Amazon ELB instance that movies.netflix.com CNAMEs to
>>in the eastern US), I have seen inconsistent answers from caching
>>resolvers for 
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com.
>>
>>Below are three different responses for
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com -
>>from Google DNS, Amazon's authoritative NS, and my local caching
>>resolver, respectively.
>>
>>You can view pcaps for these 3 at:
>>http://cloudshark.org/captures/4d24c193533b Google
>>http://cloudshark.org/captures/530a0fda5234 Amazon
>>http://cloudshark.org/captures/582e87dfda67 Local resolver
>>
>>
>>The UDP answer from Amazon has the Truncate bit set to 1, as expected. It
>>also says that there are 24 answer RRs but the UDP response contains zero
>>answers.
>>
>>This combination of behaviors seems to throw a curveball to resolvers and
>>clients alike.  You can see that the host output below says that a
>>malformed message was encountered, as does the wireshark cloudshark link
>>above for the Amazon UDP response.
>>
>>Google fails to report any AAAA answers for this name, more information
>>on that after the wall of output below.   I have looked through the
>>various RFCs pertaining to DNS a bit, but haven't found any authoritative
>>statements on the correct behavior for a properly-formed UDP response
>>packet with the truncate bit set.  So here are the questions I am left
>>with right now:
>>- Should a packet with the truncate bit set contain answers, or is this
>>optional?  I'm guessing optional, but could see arguments for the UDP
>>response with the truncate bit containing at least the first few RRs
>>- Should a packet with the truncate bit set have the field for the number
>>of Answers reflect how many answers are in that packet, or how many are
>>in the actual forthcoming response?  I believe that it should contain the
>>number of RRs contained in the UDP response itself, not the full answer
>>to the query - and this is where I believe the Amazon response is
>>malformed.  In the UDP response it says there are 24 answer RRs when
>>there are zero
>>
>>Output of host usage against these 3 servers below, with a bit more
>>information on the Google issue below
>>
>>
>>nova-dhcp-host111:~ ryan$ host -t AAAA
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com
>>8.8.8.8
>>;; Truncated, retrying in TCP mode.
>>;; communications error to 8.8.8.8#53: end of file
>>nova-dhcp-host111:~ ryan$
>>
>>nova-dhcp-host111:~ ryan$ host -t AAAA
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com
>>ns-927.amazonaws.com
>>;; Warning: Message parser reports malformed message packet.
>>;; Truncated, retrying in TCP mode.
>>Using domain server:
>>Name: ns-927.amazonaws.com
>>Address: 72.21.204.209#53
>>Aliases: 
>>
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3213:6cc8
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3211:b4fa
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3211:c04e
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3213:7430
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3213:5488
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3213:7262
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3213:6d95
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3213:6d73
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::6b14:e26c
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3211:c354
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3213:5149
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3210:fa0f
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3210:c1b2
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::ae81:f9ac
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3213:e771
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3213:f545
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3213:7747
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3213:545b
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::6b14:d04f
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3213:765d
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::6b14:fa4b
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3213:7702
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3213:722d
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3213:d9dc
>>nova-dhcp-host111:~ ryan$
>>
>>
>>nova-dhcp-host111:~ ryan$ host -t AAAA
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com
>>172.25.254.253
>>;; Truncated, retrying in TCP mode.
>>Using domain server:
>>Name: 172.25.254.253
>>Address: 172.25.254.253#53
>>Aliases: 
>>
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3213:6cc8
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3213:6d73
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3213:6d95
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3213:722d
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3213:7262
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3213:7430
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3213:765d
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3213:7702
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3213:7747
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3213:d9dc
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3213:e771
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3213:f545
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::6b14:d04f
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::6b14:e26c
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::6b14:fa4b
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::ae81:f9ac
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3210:c1b2
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3210:fa0f
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3211:b4fa
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3211:c04e
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3211:c354
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3213:5149
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3213:545b
>>dualstack.merchweb-frontend-us-999408195.us-east-1.elb.amazonaws.com has
>>IPv6 address 2406:da00:ff00::3213:5488
>>nova-dhcp-host111:~ ryan$
>>
>>
>>
>>Will Dean wanted to test the failed Google response independently of the
>>malformed Amazon response, as I was finishing up typing the above
>>message.  It looks like the EOF failure from Google is reproducible with
>>other queries that result in the truncate bit being set.
>>dnstest.managemydedi.com is set up with the intention of creating a large
>>response that results in the truncate bit being sent in the UDP response.
>>
>>nova-dhcp-host111:~ ryan$ host -t AAAA dnstest.managemydedi.com 8.8.4.4
>>;; Truncated, retrying in TCP mode.
>>;; communications error to 8.8.4.4#53: end of file
>>nova-dhcp-host111:~ ryan$
>>
>>
>>It looks like this is only broken with AAAA queries.  dns2test is packed
>>with A records, and does not cause the same problem with Google
>>
>>nova-dhcp-host111:~ ryan$ host -t A dns2test.managemydedi.com 8.8.8.8
>>;; Truncated, retrying in TCP mode.
>>Using domain server:
>>Name: 8.8.8.8
>>Address: 8.8.8.8#53
>>Aliases: 
>>
>>dns2test.managemydedi.com has address 203.0.113.0
>><bunch more answers>
>>dns2test.managemydedi.com has address 203.0.113.35
>>
>>
>