0

I've wrote me a little script to test (or extract) a very big multi-volume tar archive (note: created with GNU tar), each of which 00x.tar chunk will fit onto a standard DVD:

#!/bin/bash
d=$(dirname $0)
prf="someprefix"
last=$(ls -1 $d/*.tar | wc -l)

for i in $(eval echo {2..$last}); do echo "n $prf.00$i.tar"; done | tar -Mtvf ./$prf.001.tar --wildcards "$1"

Notes:

  • The tar pipe will work with or without the $1 parameter given.
  • The d local will help in the case when both the volume and this script are at another place, but it is run from an arbitrary directory, e. g. your $HOME.
  • The last local will work around the issue of GNU tar to never check for number of available volumes in the working directory.
  • At this enormous size, $last will hardly ever get bigger than 5 (let alone 9) - hence the 00.

OK, this works. But getting rid of the loop altogether would be even better.
Like this (--wildcards ... left out here):

 eval echo -e "n\ $d/$prf.00{2..$last}.tar\\\n" | tar -Mtvf ./$prf.001.tar

However, this only works with exactly two volumes. Set last to e. g. 4, and another issue will show up caused by echo when the pipe is omitted for a test:

n someprefix.002.tar
 n someprefix.003.tar
 n someprefix.004.tar

Ugh! How did that whitespace get in there? tar does not like that.

A "fix" (albeit very hackish) is to "sed out" the whitespace by inserting a | sed 's/^ //' into the above pipe. Quite ugly, but for heck, it works. :)

Can't the whitespace be avoided in the first place, as well?

(tr might not be a good solution here, because we'd need a substitute character (e. g. §) in order to preserve the first space after the n; save from dirty hacks, there is no way to tell tr "translate all _but_ the first space".)

  • why eval and echo -e ? – user78605 Nov 27 '14 at 10:24
  • eval is required to resolve the brace expansion: {2..$last}. echo alone does not accept a variable either as starting or ending value in braces expansion. See also post #170 here: cyberciti.biz/faq/bash-for-loop/#comment-59251 Certainly, eval can be avoided by just specifying an arbitrary value (like 100), but I did want to match last to the actual last volume. – syntaxerror Nov 27 '14 at 10:28
  • You wrote awfully lot of context for a simple question "why does eval echo -e "{2..4}\\\n" output white space at the beginning of rows starting from the 2nd row". – zagrimsan Nov 27 '14 at 11:16
  • Right in fact! :-) But by giving all that context, I wanted to avoid foolish answers like "why not omit eval and just use echo -e {2..4}'\n' instead"? It's because most people always love questioning you why you need it in that more "complicated" form and cannot use a "simpler" one (even though they might be able to think of the most obvious answer themselves: "because it's just simplified and (mostly) part of a more comprehensive script"). – syntaxerror Nov 27 '14 at 11:21
  • @syntaxerror Good to see that you got your answer, but you get downvoted just because your question, with all that specific and long context, is not very useful for someone else who might end up wondering about the same thing, since the context hides the actual question. – zagrimsan Nov 27 '14 at 14:41
2
$ eval echo -e "n\ $d/$prf.00{2..$last}.tar\\\n"
n someprefix.002.tar
 n someprefix.003.tar
 n someprefix.004.tar

How did that whitespace get in there?

The answer in a nutshell: this is analogous to the output you'd get if you echoed a line using filename expansion on a bunch of files whose names end with newline.

As described in brace expansion:

Patterns to be brace expanded take the form of an optional preamble, followed by either a series of comma-separated strings or a sequence expression between a pair of braces, followed by an optional postscript.

So the shell is evaluating

echo -e n\ dir/prefix00{2..4}.tar\\n
        ^preamble^^^^^^      ^postscript

And, as with filename expansion, it inserts a space between each expanded element, so you get (I'm keeping the \n escape sequence here for clarity):

n dir/prefix002.tar\n n dir/prefix003.tar\n n dir/prefix004.tar\n\n

QED.

As to your underlying question of how, in a script, to feed tar the info to go through multiple volumes, here are two ways:

  • Use seq to generate the n commands:
    seq -f "n $d/$prf.%03g.tar" 2 $last | tar -Mtvf $d/$prf.001.tar
  • Write a short script to generate each new volume name and pass it to tar's --new-volume-script option:
    echo '#!/bin/sh' > ./newvol
    echo "printf $d/$prf.%03d.tar \$TAR_VOLUME >&\$TAR_FD" >> ./newvol
    chmod +x ./newvol
    tar -Mtvf $d/$prf.001.tar --new-volume-script=./newvol
  • Nice you could be convinced to write a "neat" answer from your seemingly small comment (which in fact was the solution). If you agree on deleting your solution in the comments, I will do this with my reply as well, so in order to keep a better overview in comments. – syntaxerror Nov 29 '14 at 9:11
  • Ok. I was wary of just writing an answer with something rather different from what you were using, and I didn't even know if you had seq on your system, so I proposed it first in a comment. – Mark Plotnick Nov 29 '14 at 10:55
  • Oh? Just like the Red Indian tribes used to say, metaphorically: "White man always likewise cautious like his foot, ere it tread into deep water with crocodile. (Howgh!)" :-) Well you wanted to be on the safe shore, that's perfectly fine. What would this world be like if man only consisted of daredevils going like the bull at a gate. :) – syntaxerror Nov 29 '14 at 11:42
  • Your second option has a bug if your default script is not bash. You must do this instead to guarantee successful execution echo '#!/bin/bash' > ./newvol although I believe the tar info script will inherit whatever shell you use for tar by default. This is because some shells like dash do not support TAR_FD greater than 9 – jjj Jun 15 at 0:06
  • @jjj Thanks. Answer updated. – Mark Plotnick Jun 15 at 0:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.