I'm trying to copy a bunch of files with the same name, but in different subdirectories, to a single directory, changing the names to ones based on the paths to the original files. I use a for loop that does what I intend in bash, but behaves very oddly in zsh.

Command (linebreaks added for legibility):

for f in */Flavors/*/stuff1/filename.txt;
    do l=$(echo $f | cut -d'/' --output-delimiter '' -f1,3);
    echo $f $dest;
    cp -v -- $f $dest;

Output in zsh. I intend the output file names to be e.g. "EnglishAU.utf8.txt" but instead they are just "English", "French" and "Spanish". Note that the echo in the loop shows $dest containing the correct path, and then the cp uses the wrong one!

English/Flavors/AU/stuff1/filename.txt /stuff2/EnglishAU.utf8.txt
`English/Flavors/AU/stuff1/filename.txt' -> `/stuff2/English'
English/Flavors/UK/stuff1/filename.txt /stuff2/EnglishUK.utf8.txt
`English/Flavors/UK/stuff1/filename.txt' -> `/stuff2/English'
English/Flavors/US/stuff1/filename.txt /stuff2/EnglishUS.utf8.txt
`English/Flavors/US/stuff1/filename.txt' -> `/stuff2/English'
French/Flavors/CA/stuff1/filename.txt /stuff2/FrenchCA.utf8.txt
`French/Flavors/CA/stuff1/filename.txt' -> `/stuff2/French'
French/Flavors/FR/stuff1/filename.txt /stuff2/FrenchFR.utf8.txt
`French/Flavors/FR/stuff1/filename.txt' -> `/stuff2/French'
Spanish/Flavors/ES/stuff1/filename.txt /stuff2/SpanishES.utf8.txt
`Spanish/Flavors/ES/stuff1/filename.txt' -> `/stuff2/Spanish'
Spanish/Flavors/OT/stuff1/filename.txt /stuff2/SpanishOT.utf8.txt
`Spanish/Flavors/OT/stuff1/filename.txt' -> `/stuff2/Spanish'

As mentioned above, this works as intended in bash. What's zsh doing?

  • Could you please define »behaving very oddly«? What's zsh doing what you're not expecting? May 22 '14 at 21:06
  • 1
    You could use zmv. Load it with autoload -Uz zmv and use it like that: zmv -C '(*)/Flavors/(*)/stuff1.filename.txt' '/stuff2/${1}${2}.utf8.txt'. -C invokes cp instead of the default mv, source patterns surrounded by parentheses are replaced in turn by $1, $2 and so on, the patterns are alway treated as EXTENDED_GLOB patterns. See zshcontrib(1) for more information.
    – Adaephon
    May 23 '14 at 11:58
  • @Adapheon: Oh, that's really nice, pretty much exactly what I wish I had when I was working on this command. Thanks!
    – valrus
    May 23 '14 at 21:30
  • @AndreasWiese: Was my explanation between the two code boxes inadequate somehow?
    – valrus
    May 23 '14 at 21:31

This is happening because cut is outputting NULL characters in the output. You can't pass a program arguments which contain a null character (see this).

In bash this works because bash can't handle NULL characters in strings, and it strips them out. Zsh is a bit more powerful, and it can handle NULL characters. However when it comes time to pass the string to the program, it still contains the null, which signals the end of the argument.

Let's look at this in detail.

$ echo 'English/Flavors/AU/stuff1/filename.txt' | cut -d'/' --output-delimiter '' -f1,3 | xxd
0000000: 456e 676c 6973 6800 4155 0a              English.AU.

Here we simulated one of your files, passing the path through cut. Notice the xxd output which has a NULL character between English and AU.

Now lets run through and simulate the rest of the script.

$ l=$(echo 'English/Flavors/AU/stuff1/filename.txt' | cut -d'/' --output-delimiter '' -f1,3)
$ dest=/stuff2/${l}.utf8.txt
$ echo "$dest" | xxd
0000000: 2f73 7475 6666 322f 456e 676c 6973 6800  /stuff2/English.
0000010: 4155 2e75 7466 382e 7478 740a            AU.utf8.txt.

Notice the NULL after the English. The echo displays it properly because echo is a shell built-in. If we use an external echo, it also exhibits the issue.

$ /bin/echo "$dest" | xxd 
0000000: 2f73 7475 6666 322f 456e 676c 6973 680a  /stuff2/English.

P.S. You really should be quoting too :-)

The solution is to not use cut, use awk instead.

$ echo 'English/Flavors/AU/stuff1/filename.txt' | awk -F/ '{ print $1$3 }' | xxd
0000000: 456e 676c 6973 6841 550a                 EnglishAU.
  • Chapéu! Nice answer, hadn't initially thought of this. May 22 '14 at 22:17
  • Great answer, well explained. One further question, though: why are those NULLs in there? Is it because of the blank --output-delimiter option?
    – valrus
    May 23 '14 at 21:37
  • @valrus I would guess that it's deliberately coded so that if you pass an empty argument, you get a NULL delimiter. Just a guess though. Man page makes no mention of the behavior.
    – phemmer
    May 23 '14 at 21:57
  • Yes, it case like me you wonder, it's not a bug, but an undocumented feature. The source of GNU cut (--output-delimiter is GNU only) does have code specific for that and a comment that says Interpret --output-delimiter='' to mean 'use the NUL byte as the delimiter.' May 24 '14 at 21:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.