[dns-operations] [Ext] Verisign TLDs, some other servers may trim critical glue from very large referrals

Fri Jan 4 14:43:12 UTC 2019

This is an interesting protocol-implementation question.  What's being returned by the server in this case is reasonable (according to the protocol) but evidently less than useful.

The additional section is sort of a "wild-west" there aren't many formal rules for what is in there.  I've never seen a specified case where omitting a record set from the additional section would trigger the TC bit.

The tradeoff is the DS set, which is authoritatively served by the responding server, versus glue, which is not authoritative but needed for connectivity.  The two don't fit together, so one has to be dropped which is a use case that wasn't considered.  It might be good to consider which additional sets are more useful.  In this case, it is indeed the glue.

Another thing to fix is the receiving software...to know to ask for the glue when it isn't found.  But that isn't simple - you have to ask with DNSSEC off:

dig +bufsize=512 +dnssec +norecurse @b.edu-servers.net ns1.chattanoogastate.edu
(still with the large DS set)
vs.
dig +bufsize=512 +norecurse @b.edu-servers.net ns1.chattanoogastate.edu
(finally with the glue)

What a pain - to have to hunt so hard for the glue.

On 1/4/19, 08:52, "dns-operations on behalf of Matt Nordhoff" <dns-operations-bounces at dns-oarc.net on behalf of lists at mn0.us> wrote:

    Hi,

    It turns out that, with some authoritative servers, if a client sends
    a query specifying a small EDNS buffer size, and the server is
    returning a large referral with in-bailiwick nameservers, if the sizes
    align just right, some servers will trim *all* critical additional
    records from the response, *without* setting the TC bit.

    (Run-on sentence, wow.)

    If all of a parent zone's servers return such unusable referrals,
    Unbound will give up and return SERVFAIL. I don't know how other
    recursive servers behave.

    This was found in the wild by an Unbound resolver configured with an
    EDNS buffer size of 512 bytes, with the domain chattanoogastate.edu
    (TLD backend operator: Verisign). For example:

    $ dig +bufsize=512 +dnssec +norecurse @b.edu-servers.net chattanoogastate.edu

    ; <<>> DiG 9.13.5-1+ubuntu16.04.1+deb.sury.org+2-Ubuntu <<>> +bufsize
    +dnssec +norecurse @b.edu-servers.net chattanoogastate.edu
    ; (2 servers found)
    ;; global options: +cmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 53484
    ;; flags: qr; QUERY: 1, ANSWER: 0, AUTHORITY: 9, ADDITIONAL: 1

    ;; OPT PSEUDOSECTION:
    ; EDNS: version: 0, flags: do; udp: 4096
    ;; QUESTION SECTION:
    ;chattanoogastate.edu.          IN      A

    ;; AUTHORITY SECTION:
    chattanoogastate.edu.   172800  IN      NS      ns2.chattanoogastate.edu.
    chattanoogastate.edu.   172800  IN      NS      ns1.chattanoogastate.edu.
    chattanoogastate.edu.   86400   IN      DS      10114 5 2
    A22479C3577ABDDA48962F74EECCE16D3EFE14B5C95FD9463BA5A28F CD67CF3A
    chattanoogastate.edu.   86400   IN      DS      10114 5 1
    2CA51C740D54B8B3EBE5D58BD012196D5584A895
    chattanoogastate.edu.   86400   IN      DS      10618 5 1
    F8B1C75138745E23976CAD453E812D89E366E5A1
    chattanoogastate.edu.   86400   IN      DS      10618 5 2
    54592BC341F637A43C0D14F0704B58913A2B1B702266083AAEEF11B8 96E84400
    chattanoogastate.edu.   86400   IN      DS      4483 5 1
    5BC068184A5BEC46EC3C786AB8722C8E74559A3B
    chattanoogastate.edu.   86400   IN      DS      4483 5 2
    D19A8289B0EA70DF1F986138200EE3D5BDF5CA4FAEECD439C72847A6 965AF362
    chattanoogastate.edu.   86400   IN      RRSIG   DS 8 2 86400
    20190109062913 20190102051913 37217 edu.
    EncjLTtHy8zy5lHiqCe8vRayTf/8Miwik4LhUY9wkntu4uWmltruXD6J
    OiHAaOeYJeHNzlf0BYLoJCgfySL6uIlyf+3cmxobIvoGTy9bmseg+OWi
    ckkgTK5n3yU/b4w0J+j51oRGh1H2etBVxdy5a0QnUu10FIRAw6Mzi00+ tJo=

    ;; Query time: 1 msec
    ;; SERVER: 2001:503:231d::2:30#53(2001:503:231d::2:30)
    ;; WHEN: Fri Jan 04 10:57:54 UTC 2019
    ;; MSG SIZE  rcvd: 500

    Gist without mangled line wrapping:

    <https://urldefense.proofpoint.com/v2/url?u=https-3A__gist.github.com_mnordhoff_ce9f4d1a8c496751ef6e8ab435f0c45f&d=DwICAg&c=FmY1u3PJp6wrcrwll3mSVzgfkbPSS6sJms7xcl4I5cM&r=9G8-4P__AMgxNOQPiu7FkrImeieALKYtfGBE8UTuyg4&m=yJ8xMytSvOQUH1EsEoepPvEQ6SYa1yHijUedldNfhlk&s=NsAbMQ6PCEHaAlJDE4onwGQaejd4kNiiOiOtaJRjlWE&e=>

    (I'm not affiliated with Chattanooga State. They are aware of the issue.)

    I experimented with some of my own (expendable) domains under the TLDs
    .info (backend Afilias), .ooo and .xyz (both CentralNic). With them,
    _some_ of the TLD servers set TC and some do not, and I haven't been
    able to coax Unbound into failing.

    I'm running edns-buffer-size.ooo as a demonstration, with a similar
    configuration to chattanoogastate.edu.

    PowerDNS Authoritative doesn't set TC. [0] I haven't tested other DNS servers.

    As background context, with the renewed attention to fragmentation
    attacks last year, Let's Encrypt [1] and at least one other CA [2]
    have configured the resolvers in their domain validation
    infrastructure with an EDNS buffer size of 512 bytes to reduce the
    risk of fragmentation.

    Let's Encrypt changed their production DNS servers November 15. Since
    then, there have been about a half dozen public reports of problems.
    The chattanoogastate.edu issue [3] is the only one that's out of the
    ordinary -- the others were all caused by nameservers that didn't
    support TCP.

    I've chatted about this with some people; none of us had any immediate
    conclusions. (Also it's the holidays...) I haven't emailed any TLDs.

    It turns out RFC 4472 appendix B [4] discusses this problem, and it
    even links to a short namedroppers thread from 2005 [5].

    Do any of you have thoughts, experiences, quotes from standards track
    RFCs, or friends at Verisign? ;-)

    Obviously, authoritative and recursive servers _could_ both work
    around this, and people probably shouldn't set six DS records anyway.

    Should anything be changed?

    [0] <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_PowerDNS_pdns_issues_7315&d=DwICAg&c=FmY1u3PJp6wrcrwll3mSVzgfkbPSS6sJms7xcl4I5cM&r=9G8-4P__AMgxNOQPiu7FkrImeieALKYtfGBE8UTuyg4&m=yJ8xMytSvOQUH1EsEoepPvEQ6SYa1yHijUedldNfhlk&s=m84xrmDtbRp0jB9ubVHJ4-aLXEZN1V4AOGsmr0iTK2I&e=>
    [1] <https://urldefense.proofpoint.com/v2/url?u=https-3A__mn0.us_L-2Dwy&d=DwICAg&c=FmY1u3PJp6wrcrwll3mSVzgfkbPSS6sJms7xcl4I5cM&r=9G8-4P__AMgxNOQPiu7FkrImeieALKYtfGBE8UTuyg4&m=yJ8xMytSvOQUH1EsEoepPvEQ6SYa1yHijUedldNfhlk&s=ThfAT6Lr5oMKpShsZg11By-NRyrB-Xd4_QWOm4wDVtg&e=>
    <https://urldefense.proofpoint.com/v2/url?u=https-3A__community.letsencrypt.org_t_edns-2Dbuffer-2Dsize-2Dchanging-2Dto-2D512-2Dbytes_77945&d=DwICAg&c=FmY1u3PJp6wrcrwll3mSVzgfkbPSS6sJms7xcl4I5cM&r=9G8-4P__AMgxNOQPiu7FkrImeieALKYtfGBE8UTuyg4&m=yJ8xMytSvOQUH1EsEoepPvEQ6SYa1yHijUedldNfhlk&s=TjIrfufVVumABs3J_AihUdEEIkRH1s680VTk4K6os6I&e=>
    [2] <https://urldefense.proofpoint.com/v2/url?u=https-3A__mn0.us_XTNm&d=DwICAg&c=FmY1u3PJp6wrcrwll3mSVzgfkbPSS6sJms7xcl4I5cM&r=9G8-4P__AMgxNOQPiu7FkrImeieALKYtfGBE8UTuyg4&m=yJ8xMytSvOQUH1EsEoepPvEQ6SYa1yHijUedldNfhlk&s=th5AitrC1R2SSz90mZWp_fqxdD6ZD7Gbi03hKEZawcI&e=>
    <https://urldefense.proofpoint.com/v2/url?u=https-3A__groups.google.com_d_msg_mozilla.dev.security.policy_emREqhAZ3nM_ZNB35l-5FGBwAJ&d=DwICAg&c=FmY1u3PJp6wrcrwll3mSVzgfkbPSS6sJms7xcl4I5cM&r=9G8-4P__AMgxNOQPiu7FkrImeieALKYtfGBE8UTuyg4&m=yJ8xMytSvOQUH1EsEoepPvEQ6SYa1yHijUedldNfhlk&s=uRWL_0fDGLc9eqE_oNDsL3FEePsEoXHTXJ9OBUolF6A&e=>
    [3] <https://urldefense.proofpoint.com/v2/url?u=https-3A__community.letsencrypt.org_t_servfail-2Dwhile-2Drenewing_80430&d=DwICAg&c=FmY1u3PJp6wrcrwll3mSVzgfkbPSS6sJms7xcl4I5cM&r=9G8-4P__AMgxNOQPiu7FkrImeieALKYtfGBE8UTuyg4&m=yJ8xMytSvOQUH1EsEoepPvEQ6SYa1yHijUedldNfhlk&s=cFtA-kWXQHuATQ6LHso0tH0mUb7JHJrfPxdn_NpTfLU&e=>
    [4] <https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_rfc4472-23appendix-2DB&d=DwICAg&c=FmY1u3PJp6wrcrwll3mSVzgfkbPSS6sJms7xcl4I5cM&r=9G8-4P__AMgxNOQPiu7FkrImeieALKYtfGBE8UTuyg4&m=yJ8xMytSvOQUH1EsEoepPvEQ6SYa1yHijUedldNfhlk&s=jmx1Cgf4F-WWizn1GB7l7ZL4rCy4i4Ewy_lwwn6PvWc&e=>
    [5] <https://urldefense.proofpoint.com/v2/url?u=https-3A__mn0.us_Vn6R&d=DwICAg&c=FmY1u3PJp6wrcrwll3mSVzgfkbPSS6sJms7xcl4I5cM&r=9G8-4P__AMgxNOQPiu7FkrImeieALKYtfGBE8UTuyg4&m=yJ8xMytSvOQUH1EsEoepPvEQ6SYa1yHijUedldNfhlk&s=YREtU6mJUxoEDyQJWmSLJio9JJhEoJ_9Zsm7Qyh4VXY&e=>
    <https://urldefense.proofpoint.com/v2/url?u=https-3A__web.archive.org_web_20060714005533_http-3A__ops.ietf.org-3A80_lists_namedroppers_namedroppers.2005_msg01032.html&d=DwICAg&c=FmY1u3PJp6wrcrwll3mSVzgfkbPSS6sJms7xcl4I5cM&r=9G8-4P__AMgxNOQPiu7FkrImeieALKYtfGBE8UTuyg4&m=yJ8xMytSvOQUH1EsEoepPvEQ6SYa1yHijUedldNfhlk&s=Z-u1JFduCNdgHqu5P13b_MVTNbSIYlvMaVBYSzrQCGk&e=>

    Happy new year,
    -- 
    Matt Nordhoff
    _______________________________________________
    dns-operations mailing list
    dns-operations at lists.dns-oarc.net
    https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.dns-2Doarc.net_mailman_listinfo_dns-2Doperations&d=DwICAg&c=FmY1u3PJp6wrcrwll3mSVzgfkbPSS6sJms7xcl4I5cM&r=9G8-4P__AMgxNOQPiu7FkrImeieALKYtfGBE8UTuyg4&m=yJ8xMytSvOQUH1EsEoepPvEQ6SYa1yHijUedldNfhlk&s=yMVdoJl4FRO5qcbTWh1bAiiAA1Eb2cnprYYIwgl5HVE&e=
    dns-operations mailing list
    https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.dns-2Doarc.net_mailman_listinfo_dns-2Doperations&d=DwICAg&c=FmY1u3PJp6wrcrwll3mSVzgfkbPSS6sJms7xcl4I5cM&r=9G8-4P__AMgxNOQPiu7FkrImeieALKYtfGBE8UTuyg4&m=yJ8xMytSvOQUH1EsEoepPvEQ6SYa1yHijUedldNfhlk&s=yMVdoJl4FRO5qcbTWh1bAiiAA1Eb2cnprYYIwgl5HVE&e=