Manipulate file name piped from find command

Question

I'm relatively new to Bash and am trying to do something that on the surface seemed pretty straightforward - run find over a directory hierarchy to get all of the *.wma files, pipe that output to a command where I convert them to mp3 and save the converted file as .mp3. My thinking was that the command should look like the following (I've left off the audio conversion command and am instead using echo for illustration):

$ find ./ -name '*.wma' -type f -print0 | xargs -0 -I f echo ${f%.*}.mp3

As I understand it, the -print0 arg will let me handle filenames that have spaces (which many of these do as they are music files). I'm then expecting (as a result of xargs) that each file path from find is captured in f, and that using the substring match/delete from the end of the string, that I should be echoing the original file path with a mp3 extension instead of wma. However, instead of this result, I'm seeing the following:

*.mp3
*.mp3
*.mp3
*.mp3
*.mp3
*.mp3
*.mp3
*.mp3
*.mp3
...

So my question (aside from the specific 'what am I doing wrong here'), is this - do values that are the result of a pipe operation need to be treated differently in string manipulation operations than those that are the result of a variable assignment?

Answer 1

As other answers have already identified, ${f%.*} is expanded by the shell before it runs the xargs command. You need this expansion to happen once for each file name, with the shell variable f set to the file name (passing -I f doesn't do that: xargs has no notion of shell variable, it looks for the string f in the command, so if you'd used e.g. xargs -I e echo … it would have executed commands like ./somedir/somefile.wmacho .mp3).

Keeping on this approach, tell xargs to invoke a shell that can perform the expansion. Better, tell find — xargs is a largely obsolete tool and is hard to use correctly, as modern versions of find have a construct that does the same job (and more) with fewer plumbing difficulties. Instead of find … -print0 | xargs -0 command …, run find … -exec command … {} +.

find . -name '*.wma' -type f -exec sh -c 'for f; do echo "${f%.*}.mp3"; done' _ {} +

The argument _ is $0 in the shell; the file names are passed as the positional arguments which for f; do … loops over. A simpler version of this command executes a separate shell for each file, which is equivalent but slightly slower:

find . -name '*.wma' -type f -exec sh -c 'echo "${0%.*}.mp3"' {} \;

You don't actually need to use find here, assuming you're running a reasonably recent shell (ksh93, bash ≥4.0, or zsh). In bash, put shopt -s globstar in your .bashrc to activate the **/ glob pattern to recurse in subdirectories (in ksh, that's set -o globstar). Then you can run

for f in **/*.wma; do
  echo "${f%.*}.mp3"
done

(If you have directories called *.wma, add [ -f "$f" ] || continue at the beginning of the loop.)

Answer 2

In case of your solution evaluation of ${f%.*}.mp3 is done in shell that you are invoking the whole command in, not in the shell forked by the xargs. And in your shell there is no f variable, it is substituted by an empty string.

Solution using xargs with -I f:

% cat 1.sh 
#!/bin/sh
f=$1
echo ${f%.*}.mp3
% /bin/ls -1
1.sh
aaa.wma
bbbb.wma
ccccc.wma
% find ./ -name '*.wma' -type f -print0 | xargs -0 -I f ./1.sh f
./aaa.mp3
./ccccc.mp3
./bbbb.mp3

But I would do it this way:

% find ./ -name '*.wma' -type f | sed 's,\.wma$,.mp3,'