[dns-operations] Help for DNS queries in webservice

[m3047]

I am assuming "mp2" => "mod perl". I would hope, but I must ask: you're 
not making so many DNS queries you're getting rate limited, are you?

Need more facts. The facts that I can discern are:

1) Domains like .live, .club, .design, .media, ... are problematic 
compared to .com, .net, .org...

2) There are timeouts in the logs.

What causes the timeouts? Is it linked to behavior on your end? The more 
you know about your own app's behavior, the more clarity you'll have on 
your choice of mitigations. Look for patterns of queries, times of day, 
other load on (your) network, etc. If necessary write some code to improve 
this insight.

There are essentially two and a half patterns available to you for 
performing multiple synchronous (request + response) actions 
simultaneously: threading, event loop (select), forking. Forking doesn't 
really solve the problem but you can probably figure out a way to fork off 
multiple requests and then make a second pass to reap the responses.

I've never liked Perl threading. What about writing a standalone "DNS 
lookup service"? You could even make this a web service, and call it from 
your current web service or have the browser do it (ajax). Another flavor 
of this would be to write the requests to some sort of queueing system 
with a bunch of service workers and then pick up job results when they're 
ready (Redis, or two pub-sub channels (many web service workers -> many 
DNS service workers for requests, one web service worker <- many DNS 
service workers for responses).

Unless you need authoritative responses, do you run a caching DNS resolver 
(DNS server) and issue queries against that?

--

Fred Morris

On Mon, 5 Aug 2019, Paulina wrote:

> Date: Mon, 5 Aug 2019 17:59:25 +0800
> From: Paulina <pyh at geocast.net>
> To: dns-operations at lists.dns-oarc.net
> Subject: [dns-operations] Help for DNS queries in webservice
> 
> Hello,
>
> We have a mp2 handler which make a lot of DNS queries during its running 
> time. Some queris are sent to the strange domains such as .live, .club, 
> .design, .media, .digital, .market etc. I promise I never saw the domains 
> strange as those.
>
> Unlike the popular domains such as .com, .net, .org etc, which have solid DNS 
> servers and powerful networks for queries. Those small TLDs have poor 
> networks for DNS queries I may think. Thre are a lot of timeout in the logs.
>
> I have been using Net::DNS::Resolver for DNS queries in handler. Can you 
> suggest a better way to improve it? maybe AnyEvent::DNS etc?
>
> Thanks & Yours,
> Paulina
>