Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

When I don't need to adjust destination filenames I can do something like this:

$ find -type f -name '*.pat' -print0  | xargs -O cp -t /path/to/dest

It is safe because the filenames may even contain newline characters.

An alternative:

$ find -type f -name '*.pat' -print0 | cpio -p -0 -d /path/to/dest

Now I have the problem that the destination is a VFAT filesystem ... thus certain characters are just not allowed in filenames (e.g. '?'). That means that I have to adjust the destination filenames.

Something like

for i `find -type f -name '*.pat'` ; do
    cp "$i" `echo $i | sed 's/?/_/'`
done

works only for filenames without spaces - I could change IFS to just newline - but how to set '\0' as IFS?

And still - the for loop leads to as much forks/execs (of mv/sed) as you have files - which is much more excessive than the few forks/execs needed for the two examples in the beginning.

What are the alternatives to solve that problem?

share|improve this question
    
To get rid of the one process per file limit you'd have to use a special batch copy/rename application, but I wouldn't worry about spawning too many processes - the real bottleneck is very likely to be somewhere else. –  peterph Feb 24 '13 at 17:54
    
@peterph, bottleneck probably not - just perhaps a waste of resources - depending on number of files of course. Besides that - such loop-less shell constructs tend to be more concise. Cf. my answer involving GNU tar and a loop based approach. –  maxschlepzig Feb 24 '13 at 18:12
    
Could be waste of resources if you did just that and nothing else. Concise... well, yes, but at an expense - '?' is not the only character forbidden on *FAT so you can easily get rename conflicts, which you can't really resolve smoothly. –  peterph Feb 24 '13 at 18:20
add comment

3 Answers

up vote 2 down vote accepted

With pax as found on Debian, Suse, OpenBSD, NetBSD at least:

find . -type f -name '*.pat' -print0 | pax -0rws'/?/_/gp' /path/to/dest/

pax is a standard utility (contrary to tar or cpio), but its -0 option is not, though can be found in a few implementations.

If there's both a ?.pat and _.pat files, they will end up replaced with same name so one will overwrite the other in the destination. Same, if there's a _ and ? directory, their content will be merged inside the _ directory in the destination.

With GNU sort and GNU uniq, you can check for conflicts beforehand with:

find . -type f -name '*.pat' -print0 |
  tr '?' _ |
  sort -z |
  uniq -zd |
  tr '\0' '\n'

Which would report conflicting files (but not directories).

You could use zsh's zmv which would take care of conflicts, but that would still mean one mkdir and one cp per file:

autoload zmv
mkdir-and-cp() {mkdir -p -- $3:h && cp $@}
zmv -n -Qp mkdir-and-cp '(**/)*.pat(D.)' '/path/to/dest/$f:gs/?/_/'

(remove -n when happy).

share|improve this answer
add comment

One can use GNU tar for it:

$ find -type f -name '*.pat' -print0  | tar -c -f - --null --files-from - \
    | tar -C /path/to/dest -v -x -f - --show-transformed --transform 's/?/_/g'

Advantages:

  • only 3 forks/execs needed (independent of the number of files)
  • the selection of source files is very flexible - you can use the full power of (GNU) find for that (i.e. when you need to be more specific than a simple shell globbing like bash or ksh93's ** allows)
  • it is also safe for filenames containing newlines
  • the transform part is very flexible, too - for example you can use something like s/[^A-Za-z0-9 _-]/_/g which replaces all characters not in the character class [A-Za-z0-9 _-]
share|improve this answer
    
+1 Nice solution, but handling of possible filename conflicts is next to none. However without a special utility (counting those written in e.g Python), I'm afraid there is no better option than a shell loop with checking. –  peterph Feb 24 '13 at 19:40
    
@peterph, you could add --keep-old-files to the 2nd (GNU) tar command ;) –  maxschlepzig Feb 24 '13 at 22:37
    
Sure, but you usually want some kind of error message printed or even an automagic rename to happen in these caases. –  peterph Feb 24 '13 at 22:43
add comment

The best way to approach this is to define the allowed characters and bypass transposition for them instead of seeking to define characters to transpose. It is more complex than just defining a limited character set which is acceptable, because there are certain rules about what is allowed where in a filename. As such, this uses a limited subset that should be safe (as far as I can tell from reading FAT32 docs). Something like this should work (bash4+):

#!/bin/bash

shopt -s globstar nullglob

for file in **/*.pat; do
    echo cp -t "${1:-.}" "${file//[![:alnum:].\-_\/]/_}"
done

This will return something like this:

$ > bar\?.pat
$ > baz\*.pat
$ > foo.pat
$ ./script foo
cp -t baz*.pat foo/baz_.pat
cp -t bar?.pat foo/bar_.pat
cp -t foo.pat foo/foo.pat

Pass the dir to copy to as the first argument. Remove the call to echo if you want go ahead and do what it outputs.

share|improve this answer
    
Ok, IFS issue is dealt with via a **/*.pat globbing. But because of the main loop I still get one fork/exec for each file. On the one hand the recode() function eliminates an extra sed call - but on the other hand its kind of verbose - I mean too verbose for using it interactively on the command line prompt. –  maxschlepzig Feb 24 '13 at 18:17
    
Beware that bash recursive globbing follows symlinks. –  Stephane Chazelas Feb 24 '13 at 20:36
    
Note that the syntax is cp -t dest-dir files. You'd need to create the directories, remove -t and add --. Also, you can replace your recode with ${file//[!...]/_}. –  Stephane Chazelas Feb 24 '13 at 21:47
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.