From jerry at dns-oarc.net Mon Jun 1 08:57:14 2020 From: jerry at dns-oarc.net (=?UTF-8?Q?Jerry_Lundstr=c3=b6m?=) Date: Mon, 1 Jun 2020 10:57:14 +0200 Subject: [dsc] dsc-datatool v1.0.0 pre-release! In-Reply-To: References: Message-ID: <222c5aaa-fa9b-9b2c-2c4e-4409e7bf580c@dns-oarc.net> Hi all, Kind reminder that if anyone have run the new version could you please send me some feedback, even if everything just works, as it will help me move forward to the v1.0.0 release. Cheers, Jerry On 2020-04-23 13:56, Jerry Lundstr?m wrote: > Hi all, > > I am happy to announce that, thanks to funded development by EURid, > dsc-datatool v1.0.0 is now available for testing! > > This new major version is a complete rewrite of the tool, going from > Perl to Python, which have made dependency management a lot easier and > have made major updates to some of the plugins (more on that below). > > You can find the pre-release packages in our pre-release channels: > > Debian - > Ubuntu - > EPEL/Fedora - > SLE/openSUSE - > > > And man-pages exists for dsc-datatool and all plugins: > > man dsc-datatool > man dsc-datatool-generator client_subnet_country > man dsc-datatool-generator client_subnet_authority > man dsc-datatool-transformer labler > man dsc-datatool-transformer netremap > man dsc-datatool-transformer reranger > man dsc-datatool-output influxdb > > > generator: client_subnet_country > > This generator now uses Maxmind databases to look up country based on > subnets. It is recommended to install `geoipupdate` and register for > the free Geo2Lite databases at: > . See man-page for > options and more information. > > > generator: client_subnet_authority > > This generator now uses IANA's CSV data to look up the network authority > based on subnets. It can fetch these files directly from IANA or use > local files, see man-page for options and more information. > > > transformer: labler > > The previous Labler used a YAML data that was created by a contrib > script. This contrib script have also been rewritten to Python (see > `contrib` directory in source repository) and the YAML structure is the > same so you can reuse the old YAML file. It is recommended that you > update this file from time to time as new DNS types gets defined. > > > Replacing dsc-datatool v0.05 > > As many of you are probably running dsc-datatool as mentioned in the > wiki then once the new version is installed and Maxmind databases are > set up, here is the new shell part to run: > > SERVER="your-server-change-it" > NODE="your-node-change-it" > XML="your-xml-file-change-it" > dsc-datatool \ > --server "$SERVER" \ > --node "$NODE" \ > --output ";InfluxDB;file=influx.txt;dml=1;database=dsc" \ > --transform ";Labler;*;yaml=$HOME/labler.yaml" \ > --transform ";ReRanger;rcode_vs_replylen;range=/64;pad_to=5" \ > --transform ";ReRanger;qtype_vs_qnamelen;range=/16;pad_to=3" \ > --transform ";ReRanger;client_port_range;key=low;range=/2048;pad_to=5" \ > --transform > ";ReRanger;edns_bufsiz,priming_queries;key=low;range=/512;pad_to=5;allow_invalid_keys=1" > \ > --transform ";ReRanger;priming_responses;key=low;range=/128;pad_to=4" \ > --transform > ";NetRemap;client_subnet,client_subnet2,client_addr_vs_rcode,ipv6_rsn_abusers;net=8" > \ > --generator client_subnet_country \ > --generator ";client_subnet_authority;fetch=yes" \ > --xml "$XML" && > influx -import -path=influx.txt > > The wiki's () will be > updated the coming weeks with new instructions. > > > If you have any issues or bugs please report them either on GitHub, here > or email me directly. > > > > Cheers, > Jerry > _______________________________________________ > dsc mailing list > dsc at lists.dns-oarc.net > https://lists.dns-oarc.net/mailman/listinfo/dsc > From jerry at dns-oarc.net Mon Jun 1 13:31:18 2020 From: jerry at dns-oarc.net (=?UTF-8?Q?Jerry_Lundstr=c3=b6m?=) Date: Mon, 1 Jun 2020 15:31:18 +0200 Subject: [dsc] dsc v2.11.0 released! Message-ID: <64096f47-54c1-d92d-e043-ecdff5c01f64@dns-oarc.net> Hi all, This release updates the built in known TLDs table and adds the optional configuration option `knowntlds_file` to, instead of using the built in table, load the data from a file. If compiled with only MaxMindDB support then ASN and Country indexer would complain (and exit) that no database has been specified. This release changes the behavior to match that of GeoIP support, making it possible to run without specifying a database. d062e398a09f9ca4b7b81fc918657953a0fcccdff5f065cf042fb1b6557609e5 Other changes: - Fix compile warnings - COPR packaging fixes - `country_indexer`: Fixed typos in log messages (was copied from ASN) - Fix issues and false-positives reported by newer version of scan-build Packages are available at: Commits: e937d1 COPR 1382370 country, asn 423a813 scanbuild 2571b97 Compile warnings 4f69447 Known TLDs Cheers, Jerry