[dns-operations] prefetching and thundering herds

[Brian Dickson]

TL;DR:
I think the main issue is, to make sure that any caching "stubs" (e.g.
resolvers in forward-only mode) do NOT do pre-fetch, but rather only query
for expired entries when a natural query from the forwarder's client(s)
(e.g. from an application on the host) occurs.

That should, in principle, prevent thundering herd from client to recursive.

Doing the opposite (prefetch by forwarder) would definitely cause
thundering herd behavior, and likely cause significantly degraded
performance from the client applications' perspective.
(UDP, network queues, hardware queues, retries, and all that fun stuff
being triggered with greater and greater synchronization over time.)

Brian

On Wed, Jul 15, 2020 at 4:50 AM Tony Finch <dot at dotat.at> wrote:

> I've been wondering about the effects of stub resolvers with caches as
> clients of recursive servers. To what extent do they cause a thundering
> herd effect where all the cache entries expire with the same deadline?
> The herd will arrive when the RRset expires so most of those clients will
> hit maximum latency and stress the server's query deduplication mechanism.
>
> (I don't think I have enough traffic to get a useful answer from my
> servers right now.)
>
> If thundering herds happen, do they thunder enough to help explain the
> lack of benefit from prefetching observed by PowerDNS?
>
> Or maybe is the herd is too small to thunder? Instead there's a more
> gentle swell of queries after the TTL expires?
>
> https://lists.dns-oarc.net/pipermail/dns-operations/2019-April/018605.html
>
> If there is much of a herd, would it make sense to give some proportion of
> the clients a slightly reduced TTL so that they will trigger prefetch
> before the rest of them requery?
>
> Tony.
> --
> f.anthony.n.finch  <dot at dotat.at>  http://dotat.at/
> Bailey: Southwest 4 or 5, increasing 6 or 7 later. Moderate or rough,
> occasionally very rough later in far northwest. Drizzle, fog patches.
> Moderate
> or poor, occasionally very poor.
> _______________________________________________
> dns-operations mailing list
> dns-operations at lists.dns-oarc.net
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.dns-oarc.net/pipermail/dns-operations/attachments/20200715/5f8dd0f2/attachment.html>