[dns-operations] periodic DNS attacks

Thu Jan 24 17:36:52 UTC 2019

Dear DNS Operators,

Hopefully I am not completely wrong with my information to you.
Our environment: We own a class-B network. We run 4 DNS server for our domain in our own network and two additional NS are responsible at our ISP which is ACOnet.
These 4 NS which  I manage are using the latest version of BIND. Long time ago I configured rate limits on all of our NS. For log analyzing we are using a "graylog" server. Now - actually since some weeks ago - I realized a very interesting pattern looking at "graylog" for "rate limits".

[cid:image001.png at 01D4B40D.4A7B34F0]

In periodic intervals of about 16 hours I can see a very high rate of "rate limit" entries as syslog message.  This goes back to the beginning of December but  I haven't seen it the last two days.
I was very interested what happens during this period of high numbers of rate limits.
Most of the queries are for 125.147.in-addr.arpa which is our IP range.
They simple try  to figure out what names are available in our DNS DB. This is per se not an invalid activity but the intention is definitely suspect.
But stupid enough "they" do it in a high rate that it must be visible. I would never recognize such a "scan" if it is done for example each 10 seconds per query.
We are fair enough to answer over TCP if rate limit is active. So "they" got the answers.
Some others try to query some random names like wifi12345.iiasa.ac.at<http://wifi12345.iiasa.ac.at> and so on or try not IIASA domain names. Which is of course not possible with our DNS server.
So looking into the details wasn't so exciting. But still the question why it happens in so regular intervals. I wrote a small shell script collecting the graylog data from the elastic search DB in compressed and accumulated form into a sqlite3 DB.

1       2019-01-01 09:03:52.40  2019-01-01 09:13:08.64  556     58      922
2       2019-01-02 01:35:38.13  2019-01-02 01:49:18.91  820     80      1051
3       2019-01-02 16:44:25.97  2019-01-02 17:00:59.41  994     124     878
4       2019-01-03 08:55:23.21  2019-01-03 08:57:05.65  102     110     1705
5       2019-01-04 01:10:10.50  2019-01-04 01:12:28.92  138     102     1351
6       2019-01-04 17:29:27.38  2019-01-04 17:31:33.31  126     90      1862
7       2019-01-05 09:30:39.49  2019-01-05 09:33:54.40  195     150     1692
8       2019-01-05 22:10:53.11  2019-01-05 22:21:41.22  648     212     4950
9       2019-01-06 00:36:45.23  2019-01-06 00:42:21.91  336     287     3523
10      2019-01-06 16:51:41.81  2019-01-06 16:54:57.27  196     252     3501
11      2019-01-06 19:30:07.79  2019-01-06 19:32:44.99  157     5       2431
12      2019-01-07 09:12:13.40  2019-01-07 09:17:31.58  318     131     2215
13      2019-01-08 01:21:43.25  2019-01-08 01:25:24.57  221     282     2729
14      2019-01-08 17:47:07.04  2019-01-08 17:51:03.36  236     277     3334
15      2019-01-09 01:43:00.38  2019-01-09 02:12:10.43  1750    3       83317
16      2019-01-09 08:41:56.20  2019-01-09 08:47:59.56  363     258     3035
17      2019-01-09 12:13:09.20  2019-01-09 12:14:37.95  88      3       513
18      2019-01-09 22:04:59.04  2019-01-09 22:06:27.63  88      25      2137
19      2019-01-10 01:11:30.30  2019-01-10 01:15:58.38  268     318     3281
20      2019-01-10 17:29:48.95  2019-01-10 17:38:18.06  510     314     4125
21      2019-01-11 09:47:27.82  2019-01-11 09:52:05.40  278     267     2888
22      2019-01-12 02:16:10.66  2019-01-12 02:23:44.54  454     304     4870
23      2019-01-12 02:46:42.97  2019-01-12 02:49:23.98  161     163     3674
24      2019-01-12 17:11:28.92  2019-01-12 17:15:39.11  251     360     3731
25      2019-01-13 09:26:26.13  2019-01-13 09:31:00.91  274     234     3146
26      2019-01-14 01:36:36.22  2019-01-14 01:40:45.17  249     251     3672
27      2019-01-14 18:01:58.70  2019-01-14 18:07:24.72  326     226     2891
28      2019-01-15 10:17:14.05  2019-01-15 10:22:53.28  339     237     2813
29      2019-01-16 01:16:36.05  2019-01-16 01:21:00.07  264     208     3174
30      2019-01-16 13:09:09.57  2019-01-16 13:09:58.86  49      7       1445
31      2019-01-16 17:47:06.08  2019-01-16 17:51:43.31  277     296     3395
32      2019-01-16 23:35:05.84  2019-01-16 23:37:50.89  165     6       1409
33      2019-01-17 10:01:54.20  2019-01-17 10:05:53.94  239     160     3020
34      2019-01-18 00:40:44.25  2019-01-18 00:40:51.81  7       6       1218
35      2019-01-18 00:53:37.40  2019-01-18 00:53:38.87  1       6       1386
36      2019-01-18 02:21:49.23  2019-01-18 02:29:08.71  439     360     5088
37      2019-01-18 16:31:34.40  2019-01-18 16:32:32.83  58      7       1548
38      2019-01-18 17:26:14.07  2019-01-18 17:31:01.33  287     411     3828
39      2019-01-19 02:07:07.23  2019-01-19 02:07:13.77  6       6       1506
40      2019-01-19 09:26:19.13  2019-01-19 09:32:58.23  399     225     3808
41      2019-01-19 14:40:34.27  2019-01-19 14:40:35.54  1       6       1694
42      2019-01-19 17:40:01.87  2019-01-19 17:49:49.76  588     204     5775
43      2019-01-20 01:36:30.91  2019-01-20 01:41:58.13  328     317     3632
44      2019-01-20 16:56:02.77  2019-01-20 17:02:39.29  397     299     3825
45      2019-01-21 07:27:03.22  2019-01-21 07:31:42.71  279     229     3966
46      2019-01-21 10:49:33.80  2019-01-21 10:52:47.62  194     7       1619
47      2019-01-21 22:07:35.10  2019-01-21 22:08:53.06  78      260     3548
48      2019-01-22 00:00:20.36  2019-01-22 00:00:23.67  3       7       1425
49      2019-01-22 11:28:52.32  2019-01-22 11:28:52.73  0       6       1617

The table above shows that the intervals are not exactly 16 hours. It's sometimes more and sometimes less. Generally the time gap expands. Looking manually back to December it was about 15 hours. Now it's about 16.5 hours.
Definitely it is not a cronjob running anywhere. But for human activity the intervals are too identical. The increased time gap could be a result of increased numbers of IP addresses anywhere in the world involved for this activities.
The column left of the timestamp is duration in seconds. This is not an accurate value but gives a hint about time how long this attack/scan was running. In the next column we see the number IP addresses involved and then the number of events listed in graylog-server. Sequence "15" 2019-01-09 01:43 is definitely an outlier.  Also after Jan 16th there are some outlier. Under "normal" conditions only one IP address falls into this rate limit each hour. During these periods of some minutes you can see there are several hundred different IPs involved.

Another question I had in mind, are these always the same IP addresses ?

1       1305    41.9344473007712
2       735     23.6182519280206
3       365     11.7287917737789
4       279     8.96529562982005
5       161     5.17352185089974
6       88      2.82776349614396
7       48      1.54241645244216
8       36      1.15681233933162
9       28      0.89974293059126
10      12      0.38560411311054
11      1       0.032133676092545
12      2       0.06426735218509
13      3       0.096401028277635
14      11      0.353470437017995
15      14      0.44987146529563
16      11      0.353470437017995
17      6       0.19280205655527
18      7       0.224935732647815

The answer is definitely no. Above we can see that 1305 IP addresses - which is about 41.9 % - are only seen once. Only 7 IP's where logged 18 times, which is less than 1%. I record 49 such time frames of events since Jan 1st till now with total 3112 IPs. And there was not a single IP which occurred 19 times or more.  There is definitely a high number of "new" IPs never seen before.

Finally I am looking how these IP addresses are replying if queried with a DNS request. I can say that most of these IP addresses are not responding and a request is running into a timeout. These are about 80.6%. This is maybe valid, because this server was never intended to be a public available DNS server. It could also be that the system administrator recognized that there is something wrong ongoing with his server and removed it from the net. About 17.5% are replying for any query with a valid result. I asked them for a foreign domain name "www.aco.net<http://www.aco.net>" and got the correct IP. The rest is acting different.

I am still wondering what's ongoing here.
Several hundreds of IPs running into rate-limit within minutes is not normal. In the time between they're only a hand full with one or two occurrences.
Is there maybe a very simple explanation for this which I do not see ?
Or is this the background noise before a bigger DNS attack ?

Have you ever seen such a behavior ?

Kind regards
Hans

--

Ing. Dipl.-Ing. Hans Mayer
Systems Administrator
Information and Communication Technologies (ICT)

International Institute for Applied Systems Analysis (IIASA)
Schlossplatz 1
A-2361 Laxenburg, Austria
Phone: +43 2236 807 Ext 215
Mobile: +43 676 83 807 215
Web: http://www.iiasa.at
E-Mail: mayer at iiasa.ac.at<mailto:mayer at iiasa.ac.at>

Note: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID.  You may ignore it.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.dns-oarc.net/pipermail/dns-operations/attachments/20190124/0d5d921a/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 19443 bytes
Desc: image001.png
URL: <https://lists.dns-oarc.net/pipermail/dns-operations/attachments/20190124/0d5d921a/attachment.png>