11

A normal tar command

tar cvf foo.tar ./foo >foo.out 2>foo.err

has three output IO streams

  • archive data to foo.tar
  • list of filenames to STDOUT (redirected into foo.out)
  • error messages to STDERR (redirected into foo.err)

I can then inspect foo.err for error messages without having to read through the list of filenames.

if I want to do something with the archive data (pipe it through netcat or a special compression program) I can use tar's -f - option thus

tar cvf - ./foo 2>foo.err | squish > foo.tar.S

But now my list of filenames is mixed in with my error messages because tar's -v output obviously can't go to STDOUT (that's where the archive data flows) so tar cleverly writes that to STDERR instead.

Using Korn shell, is there a way to construct a command that pipes the archive stream to another command but still capture the -v output separately from any error messages.

  • Are you familiar with tee? This seems like a pretty valid use case for it. – HalosGhost Jan 28 '15 at 12:35
8

If your system supports /dev/fd/n:

tar cvf /dev/fd/3 ./foo 3>&1 > foo.out 2>foo.err | squish > foo.tar.S

Which with AT&T implementations of ksh (or bash or zsh) you could write using process substitution:

tar cvf >(squish > foo.tar.S) ./foo > foo.out 2>foo.err

That's doing exactly the same thing except that this time, the shell decides of which file descriptor to use instead of 3 (typically above 9). Another difference is that this time, you get the exit status of tar instead of squish. On systems that do not support /dev/fd/n, some shells may resort to named pipes for that feature.

If your system doesn't support /dev/fd/n or your shell can't make use of named pipes for its process substitution, that's where you'd have to deal with named pipes by hand.

5

You have to use a named pipe for that.

First create one in the folder:

mkfifo foo.pipe

Then use that command:

tar cvf foo.pipe ./foo >foo.out 2>foo.err & cat foo.pipe >foo.tar

Notice: the cat-part, can now also be gzip or whatever, that can read from a pipe:

tar cvf foo.pipe ./foo >foo.out 2>foo.err & gzip -c foo.pipe >foo.tar

Explanation:

The output is written to the name pipe (foo.pipe), where another proccess (cat, gzip, netcat) reads from. So you don't loose the stdout/stderr channels for information.

  • 1
    Might be worth noting the implications of using a named pipes. 1) best is to set a umask 077 (or use a private temp dir) to prevent other processes reading or writing to it (on many systems, named pipes, like other files are created world-readable by default), 2) you need to ensure the named pipe is used only by each instance of your script (again a one-off private temporary directory helps) 3) That means you need to clean-up afterwards or if interrupted. – Stéphane Chazelas Jan 28 '15 at 13:10
  • 1
    +1 - I like this answer. Unfortunately for me, there's some weirdness in named pipes on my (old) system that makes them unreliable when I try this. – RedGrittyBrick Jan 28 '15 at 14:48
  • 1
    @StéphaneChazelas - I think it can sometimes be easier to clean up first, like: p="/tmp/pipe$$"; mkfifo "$p"; (read na; cmd[s]...) <>"$p" & (echo;rm "$p"; cmd[s]...) >"$p" – mikeserv Jan 29 '15 at 6:01
4

GNU tar's --index-file option works well:

tar cvf - ./foo 2>foo.err --index-file=foo.out | squish > foo.tar.S

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.