11

I am trying to combine a few programs like so (please ignore any extra includes, this is heavy work-in-progress):

pv -q -l -L 1  < input.csv | ./repeat <(nc "host" 1234)

Where the source of the repeat program looks as follows:

#include <fcntl.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/epoll.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>

#include <iostream>
#include <string>

inline std::string readline(int fd, const size_t len, const char delim = '\n')
{
    std::string result;
    char c = 0;
    for(size_t i=0; i < len; i++)
    {
        const int read_result = read(fd, &c, sizeof(c));
        if(read_result != sizeof(c))
            break;
        else
        {
            result += c;
            if(c == delim)
                break;
        }
    }
    return result;
}

int main(int argc, char ** argv)
{
    constexpr int max_events = 10;

    const int fd_stdin = fileno(stdin);
    if (fd_stdin < 0)
    {
        std::cerr << "#Failed to setup standard input" << std::endl;
        return -1;
    }


    /* General poll setup */
    int epoll_fd = epoll_create1(0);
    if(epoll_fd == -1) perror("epoll_create1: ");
    {
        struct epoll_event event;
        event.events = EPOLLIN;
        event.data.fd = fd_stdin;
        const int result = epoll_ctl(epoll_fd, EPOLL_CTL_ADD, fd_stdin, &event);
        if(result == -1) std::cerr << "epoll_ctl add for fd " << fd_stdin << " failed: " << strerror(errno) << std::endl;
    }

    if (argc > 1)
    {
        for (int i = 1; i < argc; i++)
        {
            const char * filename = argv[i];
            const int fd = open(filename, O_RDONLY);
            if (fd < 0)
                std::cerr << "#Error opening file " << filename << ": error #" << errno << ": " << strerror(errno) << std::endl;
            else
            {
                struct epoll_event event;
                event.events = EPOLLIN;
                event.data.fd = fd;
                const int result = epoll_ctl(epoll_fd, EPOLL_CTL_ADD, fd, &event);
                if(result == -1) std::cerr << "epoll_ctl add for fd " << fd << "(" << filename << ") failed: " << strerror(errno) << std::endl;
                else std::cerr << "Added fd " << fd << " (" << filename << ") to epoll!" << std::endl;
            }
        }
    }

    struct epoll_event events[max_events];
    while(int event_count = epoll_wait(epoll_fd, events, max_events, -1))
    {
        for (int i = 0; i < event_count; i++)
        {
            const std::string line = readline(events[i].data.fd, 512);                      
            if(line.length() > 0)
                std::cout << line << std::endl;
        }
    }
    return 0;
}

I noticed this:

  • When I just use the pipe to ./repeat, everything works as intended.
  • When I just use the process substitution, everything works as intended.
  • When I encapsulate pv using process substitution, everything works as intended.
  • However, when I use the specific construction, I appear to lose data (individual characters) from stdin!

I have tried the following:

  • I have tried to disable buffering on the pipe between pv and ./repeat using stdbuf -i0 -o0 -e0 on all processes, but that doesn't seem to work.
  • I have swapped epoll for poll, doesn't work.
  • When I look at the stream between pv and ./repeat with tee stream.csv, this looks correct.
  • I used strace to see what was going on, and I see lots of single-byte reads (as expected) and they also show that data is going missing.

I wonder what is going on? Or what I can do to investigate further?

16

Because the nc command inside <(...) will also read from stdin.

Simpler example:

$ nc -l 9999 >/tmp/foo &
[1] 5659

$ echo text | cat <(nc -N localhost 9999) -
[1]+  Done                    nc -l 9999 > /tmp/foo

Where did the text go? Through the netcat.

$ cat /tmp/foo
text

Your program and nc compete for the same stdin, and nc gets some of it.

  • You're right! Thanks! Can you suggest a clean way to disconnect stdin in the <(...)? Is there a nicer way than <( 0<&- ...)? – Roel Baardman Jun 10 at 8:55
  • 5
    <(... </dev/null). don't use 0<&-: it will cause the first open(2) to return 0 as the new fd. If your nc supports it, you can also use the -d option. – mosvy Jun 10 at 8:56
3

epoll() or poll() returning with E/POLLIN will only tell you that a single read() may not block.

Not that you will be able to do a lot of one byte read()s up to a newline, as you do.

I say may because a read() after epoll() returned with E/POLLIN may still block.

Your code will also try to read past EOF, and completely ignores any read() errors.

  • Despite this not being a direct solution to my problem, thanks for making the comment. I realize that this code has flaws, and EOF detection is present in a less stripped-down version (through use of POLLHUP/POLLNVAL). I do struggle with finding an unbuffered way to read lines from multiple file descriptors though. My repeat program is essentially processing NMEA data (line-based and with no length indicators) from multiple sources. Since I'm combining data from multiple live sources, I'd like my solution to be unbuffered. Can you suggest a more efficient way to do this? – Roel Baardman Jun 10 at 8:59
  • fwiw, doing a system call (read) for each byte is the least efficient way possible. EOF checking can be done by just checking the return value of read, no need for POLLHUP (and POLLNVAL will only be returned when you pass it a bogus fd, not on EOF). But anyways, stay tuned. I have the idea of a ypee utility which reads from multiple fds and mix them into another fd, while preserving records (keeping lines intact). – pizdelect Jun 10 at 21:55
  • I noticed that this bash construction should do that, but I don't know how to combine stdin in it: { cmd1 & cmd2 & cmd3; } > file File will contain what you describe. However, in my case I'm running everything from tcpserver(3), so I want to include stdin (which contains the client data) as well. I'm not sure how to do that. – Roel Baardman Jun 12 at 5:34
  • 1
    It depends on what cmd1, cmd2, ... are. If they're nc or cat and your data is line-oriented, the output may be malformed -- you'll get lines consisting of the start of a line printed by cmd1 and the end of a line printed by cmd2. – pizdelect Jun 12 at 12:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.