[dsc] dsp multi node setup: no data to display at this time

Wed Apr 24 11:28:05 UTC 2019

Hi Jerry

See my comments inline:

-----Original Message-----
From: Jerry Lundström <jerry at dns-oarc.net> 
Sent: Wednesday, April 24, 2019 11:36 AM
To: Mächler Philippe <P.Maechler at glattwerk.ch>
Cc: dsc at dns-oarc.net
Subject: Re: [dsc] dsp multi node setup: no data to display at this time

Hi Philippe,

>On 4/24/19 11:09 AM, Mächler Philippe wrote:
>> The xml Files are processed by refile-and-grok.sh
>> # tail dsc-xml-extractor.out
>> wrote 17 lines to 20190424/client_port_range.dat
>> wrote 17 lines to 20190424/ipv6_rsn_abusers_count.dat
>> wrote 0 lines to 20190424/ipv6_rsn_abusers_accum.dat
>> wrote 17 lines to 20190424/qr_aa_bits.dat
>> wrote 955 lines to 20190424/client_addr_vs_rcode_accum.dat
>> wrote 2 lines to 20190424/chaos_types_and_names.dat
>> wrote 17 lines to 20190424/transport_vs_qtype.dat
>> wrote 17 lines to 20190424/pcap_stats.dat
>> wrote 17 lines to 20190424/rcode.dat
>> wrote 17 lines to 20190424/direction_vs_ipproto.dat
>
>This looks a bit light, what dsc.conf are you running?
>

Here is my dsc.conf, i removed some comments
# cat /usr/local/etc/dsc/dsc.conf
# local_address
#
local_address 127.0.0.1;
local_address ::1;
local_address 192.168.3.221;
local_address X.Y.192.58;
local_address X.Y.192.80;

# run_dir
run_dir "/var/dsc";

# minfree_bytes
minfree_bytes 5000000;

# pid_file
pid_file "/var/run/dsc.pid";

# bpf_program
#
#  use this to see only DNS messages
#bpf_program "udp port 53";
#
#  use this to see only DNS *queries*
#bpf_program "udp dst port 53 and udp[10:2] & 0x8000 = 0";

# pcap_buffer_size
#
#pcap_buffer_size 4194304;

# pcap_thread_timeout
#
#pcap_thread_timeout 100;

# drop_ip_fragments
#drop_ip_fragments;

# interface
interface bge0;
interface bge1;
interface lo0;
interface lo1;
#interface /path/to/dump.pcap;

# qname_filter
#qname_filter WWW-Only ^www\. ;
#dataset qtype dns All:null Qtype:qtype queries-only,WWW-Only ;

# datasets
#
#  please see the DSC manual for more information.
dataset qtype dns All:null Qtype:qtype queries-only;
dataset rcode dns All:null Rcode:rcode replies-only;
dataset opcode dns All:null Opcode:opcode queries-only;
dataset rcode_vs_replylen dns Rcode:rcode ReplyLen:msglen replies-only;
dataset client_subnet dns All:null ClientSubnet:client_subnet queries-only max-cells=200;
dataset qtype_vs_qnamelen dns Qtype:qtype QnameLen:qnamelen queries-only;
dataset qtype_vs_tld dns Qtype:qtype TLD:tld queries-only,popular-qtypes max-cells=200;
dataset certain_qnames_vs_qtype dns CertainQnames:certain_qnames Qtype:qtype queries-only;
dataset client_subnet2 dns Class:query_classification ClientSubnet:client_subnet queries-only max-cells=200;
dataset client_addr_vs_rcode dns Rcode:rcode ClientAddr:client replies-only max-cells=50;
dataset chaos_types_and_names dns Qtype:qtype Qname:qname chaos-class,queries-only;
#dataset country_code dns All:null CountryCode:country queries-only;
#dataset asn_all dns IPVersion:dns_ip_version ASN:asn queries-only max-cells=200;
dataset idn_qname dns All:null IDNQname:idn_qname queries-only;
dataset edns_version dns All:null EDNSVersion:edns_version queries-only;
dataset edns_bufsiz dns All:null EDNSBufSiz:edns_bufsiz queries-only;
dataset do_bit dns All:null D0:do_bit queries-only;
dataset rd_bit dns All:null RD:rd_bit queries-only;
dataset idn_vs_tld dns All:null TLD:tld queries-only,idn-only;
dataset ipv6_rsn_abusers dns All:null ClientAddr:client queries-only,aaaa-or-a6-only,root-servers-net-only max-cells=50;
dataset transport_vs_qtype dns Transport:transport Qtype:qtype queries-only;
dataset client_port_range dns All:null PortRange:dns_sport_range queries-only;
#dataset second_ld_vs_rcode dns Rcode:rcode SecondLD:second_ld replies-only max-cells=50;
#dataset third_ld_vs_rcode dns Rcode:rcode ThirdLD:third_ld replies-only max-cells=50;
dataset direction_vs_ipproto ip Direction:ip_direction IPProto:ip_proto any;
#dataset dns_ip_version_vs_qtype dns IPVersion:dns_ip_version Qtype:qtype queries-only;

#  datasets for collecting data on priming queries at root nameservers
#dataset priming_queries dns Transport:transport EDNSBufSiz:edns_bufsiz priming-query,queries-only;
#dataset priming_responses dns All:null ReplyLen:msglen priming-query,replies-only;

#   dataset for monitoring an authoritative nameserver for DNS reflection attack
dataset qr_aa_bits dns Direction:ip_direction QRAABits:qr_aa_bits any;

# dataset for servfail response for dnssec validation fail.
#dataset servfail_qname dns ALL:null Qname:qname servfail-only,replies-only;

# dataset for successful validation.
#dataset ad_qname dns ALL:null Qname:qname authentic-data-only,replies-only;

# bpf_vlan_tag_byte_order
#bpf_vlan_tag_byte_order host;

# match_vlan
#match_vlan 100 200;

# statistics_interval
#statistics_interval 60;

# no_wait_interval
#no_wait_interval;

# output_format
#output_format XML;
#output_format JSON;

# dump_reports_on_exit
#dump_reports_on_exit;

# geoip
#geoip_v4_dat "/usr/share/GeoIP/GeoIP.dat" STANDARD MEMORY_CACHE MMAP_CACHE;
#geoip_v6_dat "/usr/share/GeoIP/GeoIPv6.dat";
#geoip_asn_v4_dat "/usr/share/GeoIP/GeoIPASNum.dat" MEMORY_CACHE;
#geoip_asn_v6_dat "/usr/share/GeoIP/GeoIPASNumv6.dat" MEMORY_CACHE;

# Client Subnet Mask
#client_v4_mask 255.255.255.0;
#client_v6_mask ffff:ffff:ffff:ffff:ffff:ffff:0000:0000;

This is the default config, apart from the local_address and interface parameter

>
>> # less /var/log/dsc.log
>> [14317] ===> starting at Tue Apr 23 17:25:40 CEST 2019
>> [14317] ARGS=$VAR1 = {
>>           'content' => 'png',
>>           'window' => 14400,
>>           'plot' => 'bynode',
>>           'end' => 1556033140,
>>           'binsize' => 60,
>>           'server' => 'none',
>>           'yaxis' => 'rate',
>>           'node' => 'all',
>>           'mini' => 0
>>         };
>> 
>> [14317] Plotting none all bynode 1556033140 14400 60
>> [14317] reading datafile took 0 seconds, 0 lines
>> [14317] loaded data: $VAR1 = {};
>> 
>> [14317] munged data: $VAR1 = {};
>> 
>> [14317] writing trace tmpfile took 0 seconds, 0 lines
>> [14317] graph took 0 seconds
>> [14317] <=== finished at Tue Apr 23 17:25:40 CEST 2019
>
>Hmm, this didn't generate any graph because there was no data.
>
>Can you open a graph with data so we see it loads it and calls ploticus
>correctly?

I'm note sure what exactly you mean. There is no graph to open :(

>> The .dat Files on /var/lib/dsc/$SERVER/$NODE belong to root:wheel with read permissions for everyone, when I changed them to www:www the effect is the same ☹
>
>Did you setup the Presenter cache also?

I created the directory and set the permissions to the same user as nginx is running, but I haven't seen a config option to specify the cache dir. 

pmaechler at DSP:/ # ll /var/cache/
total 1
drwxr-xr-x  2 www   www    2 Apr 23 15:13 dsp/
drwxr-xr-x  2 root  wheel  4 Mar 29  2018 pkg/
pmaechler at DSP:/ # ll /var/cache/
dsp/ pkg/ 
pmaechler at DSP:/ # ll /var/cache/dsp/
total 0
pmaechler at DSP:/ # 

If I grep throug the files I see the same cachepath

pmaechler at DSP:/ # grep cache /usr/local/lib/cgi-bin/*
/usr/local/lib/cgi-bin/dsc-grapher.pl:    cachepath => '/var/cache/dsp',
pmaechler at DSP:/ # grep cache /usr/local/libexec/dsp/*
/usr/local/libexec/dsp/dsc-grapher-cli.pl:    'cachepath=s',
/usr/local/libexec/dsp/dsc-grapher-cli.pl:    cachepath => $args{cachepath} ? $args{cachepath} : '/var/cache/dsp',
pmaechler at DSP:/ #

Best regards
Philippe