[dns-operations] Open source release of the DNS-STATS Compactor

[Jim Hague]

On 21/06/2017 17:28, Robert Edmonds wrote:
> Jim Hague wrote:
>> Yes, I did verify that it's not a libtins problem. I reproduced the
>> problem with a small libpcap-only program. However, I think that it is
>> in fact a kernel problem rather than libpcap.
>>
>> Also, it's only a problem when using pacp_next()/pcap_next_ex() or
>> pcap_dispatch(). It doesn't manifest with pcap_loop(), which is why I
>> think tcpdump works.
> 
> Any chance you could share that libpcap test program? That sounds like a
> pretty serious bug. The Debian Jessie release is still supported by the
> Debian project, so if that problem is due to a kernel or libpcap bug it
> would be nice to get it fixed in an update.

No problem. Attached. I'm building with:
$ g++ -std=c++11 -o test_pcap test_pcap.cpp -lpcap

When run on a 14.04.5 install with the linux-image-generic kernel, as below:

jim at trusty:~$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 14.04.5 LTS
Release:	14.04
Codename:	trusty
jim at trusty:~$ uname -a
Linux trusty 3.13.0-121-generic #170-Ubuntu SMP Wed Jun 14 09:04:33 UTC
2017 x86_64 x86_64 x86_64 GNU/Linux

or a Debian Jessie install:

jim at jim-dev-debian:~/Develop/src/dns-cap-cpp/src$ lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 8.8 (jessie)
Release:        8.8
Codename:       jessie
jim at jim-dev-debian:~/Develop/src/dns-cap-cpp/src$ uname -a
Linux jim-dev-debian 3.16.0-4-amd64 #1 SMP Debian 3.16.43-2+deb8u1
(2017-06-18) x86_64 GNU/Linux

$ sudo ./test_pcap loop <interface>

will show packets being received as expected from external input, e.g.

$ ping -c 5 -i 0.2 <jessie host>

Re-run the test as

$ sudo ./test_pcap dispatch <interface>

and you won't see the expected number of packets. It looks to me that if
there's more a gap of about 50 milliseconds or more between packets,
you'll lose the first packet after the gap.

Judging by the timing of the printed output, a waiting select() on the
PCAP handle exits on receipt of a new packet, but a subsequent call to
to pcap_next_ex() or pcap_dispatch() does not find the packet.

A fresh 14.04.5 install will install linux-image-generic-lts-xenial, not
linux-image-generic. I believe this has been the case since 14.04.2. I
see no problems with this kernel series, or with Xenial installs.
-- 
Jim Hague - jim at sinodun.com          Never trust a computer you can't lift.
-------------- next part --------------
#include <cstdio>
#include <cstdlib>
#include <iostream>

#include <pcap/pcap.h>
#include <sys/select.h>

void error(const char* msg)
{
    std::perror(msg);
    std::exit(1);
}

void usage()
{
    std::cerr << "Usage: test_pcap [-n|--non-blocking] next|dispatch|loop <interface>\n";
    std::exit(1);
}

void dispatcher(u_char *user, const struct pcap_pkthdr* h, const u_char* bytes)
{
    static int no_read = 0;
    std::cout << "Read packet " << ++no_read << std::endl;
}

int main(int ac, char *av[])
{
    bool non_blocking = false;

    if ( ac < 3 || ac > 4 )
        usage();

    ac--;
    av++;

    std::string arg(*av);

    if ( arg == "-n" || arg == "--non-blocking" )
    {
        non_blocking = true;
        ac--;
        av++;

        if ( ac < 2 )
            usage();
        arg = *av;
    }

    char errbuf[PCAP_ERRBUF_SIZE];
    pcap_t* handle = pcap_create(av[1], errbuf);
    if ( !handle )
        error(av[1]);

    if ( pcap_set_snaplen(handle, 65535) != 0 )
        error("snaplen");

    if ( pcap_activate(handle) < 0 )
        error("activate");

    if ( non_blocking && pcap_setnonblock(handle, 1, errbuf) < 0 )
        error("nonblock");

    int fd = pcap_get_selectable_fd(handle);
    if ( fd < 0 )
        error("selectable");

    fd_set fdset;
    FD_ZERO(&fdset);
    FD_SET(fd, &fdset);

    for(;;)
    {
        if ( arg == "next" )
        {
            struct pcap_pkthdr* hdr;
            const u_char* data;
            bool read_one;

            do
            {
                read_one = false;

                std::cout << "Reading..." << std::endl;
                switch (pcap_next_ex(handle, &hdr, &data))
                {
                case 1:
                    read_one = true;
                    dispatcher(nullptr, nullptr, nullptr);
                    continue;

                case 0:
                    std::cout << "Nothing read" << std::endl;
                    break;

                default:
                    error("packet read");
                    break;
                }
            } while ( read_one );
        }
        else if ( arg == "dispatch" )
        {
            std::cout << "Dispatching..." << std::endl;
            pcap_dispatch(handle, -1, dispatcher, nullptr);
        }
        else if ( arg == "loop" )
        {
            std::cout << "Looping..." << std::endl;
            pcap_loop(handle, -1, dispatcher, nullptr);
        }
        else
            usage();

        fd_set fd_selected = fdset;
        struct timeval tv;
        tv.tv_sec = 2;
        tv.tv_usec = 0;
        switch (select(fd + 1, &fd_selected, nullptr, nullptr, &tv))
        {
        case -1:
            switch(errno)
            {
            case EAGAIN:
            case EINTR:
                break;

            default:
                error("select");
                break;
            }

        default:
            break;
        }
    }
}