7

This question already has an answer here:

I have a read-only file, F.

A program, P, that I'm not the author of, needs to read F.

I want the content of F to come from another 'generator' program, G, whenever P tries to read F (taking F to be an ordinary file) and not any earlier.

I tried doing the following:

$ mkfifo /well-known/path/to/F    # line #1
$ G > /well-known/path/to/F       # line #2

Now, when P starts up and tries to read F, it appears to be able to read the output generated by G just as I wished it to. However, it can do so only once, since G after all gets to run only once! So, if P had a need to read F again later in its execution, it would end up blocking on the fifo!

My question is, other than bracketing line #2 above in some sort of an infinite loop, is there any other (elegant) alternative for the above?

What I'm hoping for is, some way of registering a 'hook' program into the file-open system call such that the file-open would result in the launching of the hook-program and the file-read in the reading of the hook-program output. Obviously the assumption here is: the read will happen sequentially from file beginning to file end, and never in random seeks.

marked as duplicate by Gilles 'SO- stop being evil' files Apr 2 '16 at 20:20

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

6

FUSE + a soft-link (or a bind mount) is a solution, though I would not consider it "elegant", there's quite a lot of baggage. On *BSD you'd have the simpler option of portalfs, with which you could solve the problem with a symlink – there was a port of it to Linux many years ago, but it seems to have been dropped, presumably in favour of FUSE.

You can quite easily inject a library to override the required open()/open64() libc call(s) that it makes. e.g.:

#define _GNU_SOURCE
#include <stdio.h>
#include <fcntl.h>
#include <string.h>
#include <dlfcn.h>
#include <stdarg.h>

//  gcc -Wall -rdynamic -fPIC -nostartfiles -shared -ldl -Wl,-soname,open64 \
//       -o open64.so open64.c

#define DEBUG 1
#define dfprintf(fmt, ...) \
    do { if (DEBUG) fprintf(stderr, "[%14s#%04d:%8s()] " fmt, \
          __FILE__, __LINE__, __func__, __VA_ARGS__); } while (0)

typedef int open64_f(const char *pathname, int flags, ...);
typedef int close_f(int fd);
static  open64_f   *real_open64;
static  close_f    *real_close;

static FILE *mypipe=NULL;
static int mypipefd=-1;

//void __attribute__((constructor)) my_init()
void _init()
{
    char **pprog=dlsym(RTLD_NEXT, "program_invocation_name");
    dfprintf("It's alive! argv[0]=%s\n",*pprog);

    real_open64 = dlsym(RTLD_NEXT, "open64");
    dfprintf("Hook %p open64()\n",(void *)real_open64);
    if (!real_open64) printf("error: %s\n",dlerror());

    real_close = dlsym(RTLD_NEXT, "close");
    dfprintf("Hook %p close()\n",(void *)real_close);
    if (!real_close) printf("error: %s\n",dlerror());
}

int open64(const char *pathname, int flags, ...)
{
    mode_t tmpmode=0;
    va_list ap;
    va_start(ap, flags);
    if (flags & O_CREAT) tmpmode=va_arg(ap,mode_t);
    va_end(ap);

    dfprintf("open64(%s,%i,%o)\n",pathname,flags,tmpmode);

    if (!strcmp(pathname,"/etc/passwd")) {
        mypipe=popen("/usr/bin/uptime","r");
        mypipefd=fileno(mypipe);
        dfprintf("  popen()=%p fd=%i\n",mypipe,mypipefd);
        return mypipefd;
    } else {
        return real_open64(pathname,flags,tmpmode);
    }
}

int close(int fd)
{
    int rc;
    dfprintf("close(%i)\n",fd);
    if (fd==mypipefd) {
        rc=pclose(mypipe); // pclose() returns wait4() status
        mypipe=NULL; mypipefd=-1;
        return (rc==-1) ? -1 : 0;
    } else  {
        return real_close(fd);
    }
}

Compile and run:

$ gcc -Wall -rdynamic -fPIC -nostartfiles -shared -ldl -Wl,-soname,open64   \
    -o open64.so open64.c 
$ LD_PRELOAD=`pwd`/open64.so cat /etc/passwd
19:55:36 up 1110 days,  9:19, 55 users,  load average: 0.53, 0.33, 0.29

Depending on exactly how the application works (libc calls), you may need to handle open() or fopen()/fclose() instead. The above works for cat or head, but not sort since it calls fopen() instead (it's straightforward to add fopen()/fclose() to the above too).

You probably need more error handling and sanity checking than the above (especially with a long running program, to avoid leaks). This code does not correctly handle concurrent opens.

Since a pipe and a file have obvious differences, there is a risk that the program will malfunction.

Otherwise, assuming you have daemon and socat you can pretend you don't have an infinite loop:

daemon -r -- socat -u EXEC:/usr/bin/uptime PIPE:/tmp/uptime    

This has the slight disadvantage (which should be evident here) of the provider program starting to write then blocking, so you see an old uptime, instead of it being run on-demand. Your provider would need to use non-blocking I/O in order to properly provide just-in-time data. (A unix domain socket would allow a more conventional client/server approach, but that's not the same as a FIFO/named pipe that you can just drop in.)


Update see also this later question which covers the same topic, though generalised to arbitrary processes/readers rather than a specific one: How can I make a special file that executes code when read from (note that fifo-only answers won't reliably generalise to concurrent reads)

  • I'm tempted to make your answer final. Will wait for a few more days to see any further responses or comments from others. Thanks, and +1, meanwhile. – Harry May 24 '13 at 2:48
  • Your answer is, essentially, the same as Hauke's. But because yours has more detail in it (which must have required more effort on your part), I'm flagging yours as final. – Harry May 26 '13 at 9:39
4

What you could do is having a daemon (let's call it filld as it fills up F each time P needs it).

When you launch it, it tries to open the FIFO (and blocks as there is no reader). Each time a reader comes, it writes to the FIFO what it needs to write (fork-ing and exec-ing G for example), closes the FIFO and reopens it. Each time, it succeeds in opening, it writes to F. If it receives a SIGPIPE, it closes everything, and blocks itself on the FIFO again.

  • But how is this different from shell-based infinite loop I proposed in the original question above? – Harry May 20 '13 at 14:47
  • 1
    It can be managed by openrc or serviced (or whatever your distribution has to manage init scripts) for example. – lgeorget May 20 '13 at 14:52
  • Thanks. I'm upvoting your response and comment in any case. – Harry May 20 '13 at 15:05
2

An infinite loop is the reason for at least two reasons. Beside the one you noticed: You want the file updated on every access, don't you? Otherwise you could simply create a regular file with the right content.

A hook for open*() is possible but probably not trivial. FUSE is the way to go. If you are lucky then a module for this already exists. Otherwise you need to write your own module which hooks this one path and passes the rest through.

  • So, with the FUSE approach, I'd have to point P's use of F to the FUSE file system somehow... which could be either via (soft-)linking, or mounting, or...? Is that right? Have never dabbled with FUSE before. it would be an interesting thing to play with. – Harry May 20 '13 at 14:54
  • 1
    @Harry FUSE creates a whole file system which must be mounted somewhere to work. – Hauke Laging May 20 '13 at 14:57
  • I may or may not go this route but I do appreciate your mentioning FUSE. – Harry May 20 '13 at 15:11

Not the answer you're looking for? Browse other questions tagged or ask your own question.