0

The first and second invocation are for comparison's sake. It's the third I'm trying to get to work.

$ ls -a1 *mp3
DaftPunk_VeridisQuo.mp3
French79_AfterParty.mp3
French79_BetweentheButtons.mp3
French79_Hometown.mp3
$ find . -maxdepth 1 -type f -name '*mp3'
./French79_AfterParty.mp3
./French79_Hometown.mp3
./DaftPunk_VeridisQuo.mp3
./French79_BetweentheButtons.mp3
$ for x in "$(ls -a *mp3)"; do mv "./$x" "./electro_$x"; done
mv: cannot stat './DaftPunk_VeridisQuo.mp3'$'\n''French79_AfterParty.mp3'$'\n''French79_BetweentheButtons.mp3'$'\n''French79_Hometown.mp3': No such file or directory
2
8

I prefer to use it directly:

for x in *.mp3
do 
    mv ./"$x"  "electro_$x"
done
3
  • @steeldriver Thank you (no more wine to my glass ☺). Just one question: what is the advantage of ./"$x" vs "$x" ? – JJoao Oct 21 '19 at 7:16
  • 2
    It's a way to prevent the filename from being parsed as a list of options in the case that it begins with a dash - you might want to change it to mv -- "$x" "electro_$x" which is probably more idiomatic (I couldn't decide... now I'm leaning towards the latter) – steeldriver Oct 21 '19 at 11:45
  • This is the idiomatic way to do it either way -- shouldn't you also show the paramater expansion for more complex renaming rules? Like replacing, not just adding to, the filename? And then? ecxecpt null and slash, everthing works? – user373503 Oct 21 '19 at 13:58
3

By quoting the $(..), you get one token and not N tokens:

I start out with a couple sample files:

$ ls
a.mp3 b.mp3 c.mp3

If I do what you did, I get one line with all three:

for i in "$(ls *.mp3)"; do
    echo "--> $i"
done
--> a.mp3 b.mp3 c.mp3

If I omit the quotes around the $(...), I get three different output lines:

for i in $(ls *.mp3); do
    echo "--> $i"
done
--> a.mp3
--> b.mp3
--> c.mp3

If you have files with spaces, then something like this might solve your problem (note that this applies only to files in the current directory):

Before

$ ls
'DaftPunk VeridisQuo.mp3'  'French79 AfterParty.mp3'  'French79 BetweentheButtons.mp3'  'French79 Hometown.mp3'

Use find to do the rename:

$ find *.mp3 -maxdepth 1 -type f -name *.mp3 -exec mv {} "electro_{}" \;
$ ls
'electro_DaftPunk VeridisQuo.mp3'  'electro_French79 AfterParty.mp3'  'electro_French79 BetweentheButtons.mp3'  'electro_French79 Hometown.mp3'

Why did I suggest find *.mp3 and not simply find . -type f -name '*.mp3' ...?

$ find . -maxdepth 1 -type f -name '*.mp3' -exec mv {} "electro_{}" \;
mv: cannot move './French79 Hometown.mp3' to 'electro_./French79 Hometown.mp3': No such file or directory
mv: cannot move './French79 BetweentheButtons.mp3' to 'electro_./French79 BetweentheButtons.mp3': No such file or directory
mv: cannot move './French79 AfterParty.mp3' to 'electro_./French79 AfterParty.mp3': No such file or directory
mv: cannot move './DaftPunk VeridisQuo.mp3' to 'electro_./DaftPunk VeridisQuo.mp3': No such file or directory
4
  • OK. The reason for quoting must have been that there were spaces in the original filenames. After restoring them, and using your approach. $ for x in $(ls -a *mp3); do mv "./$x" "./electro_$x"; done returns: mv: cannot stat './DaftPunk': No such file or directory etc. – Erwann Oct 20 '19 at 23:54
  • 2
    Yes, using ls for this is fragile (because of things like spaces); I'm guessing that's what you had also considered find. find is a better option. – Andy Dalton Oct 21 '19 at 0:04
  • "If you have files with spaces, then something like this might solve your problem" Yes, but I was toying with ls. I'll accept the answer if no one shows a solutions with spaces and ls. – Erwann Oct 21 '19 at 1:38
  • 1
    find *.mp3 ... is NOT how you use find - the *.mp3 should be one-or-more starting directories, and it's a needless duplication of the -name predicate. Try find . -maxdepth 1 -type -f -name '*.mp3' -exec mv {} "electro_{}" \; – cas Oct 21 '19 at 3:25
2

What you're trying to do is a simple bulk-rename that is easily handled by the perl rename utility (aka prename or file-rename). This is NOT the same as the rename utility in the util-linux package (which has completely different and incompatible command-line options and capabilities).

Try

rename -n 's/^/electro_/' *.mp3

The -n option makes this a dry-run that will only show you how the .mp3 files would be renamed if you let it. To actually rename them, either remove the -n or replace it with -v for verbose output.

$ touch DaftPunk_VeridisQuo.mp3 French79_AfterParty.mp3 French79_BetweentheButtons.mp3

$ ls -l
total 2
-rw-r--r-- 1 cas cas 0 Oct 21 14:30 DaftPunk_VeridisQuo.mp3
-rw-r--r-- 1 cas cas 0 Oct 21 14:30 French79_AfterParty.mp3
-rw-r--r-- 1 cas cas 0 Oct 21 14:30 French79_BetweentheButtons.mp3

$ rename -v 's/^/electro_/' *.mp3
DaftPunk_VeridisQuo.mp3 renamed as electro_DaftPunk_VeridisQuo.mp3
French79_AfterParty.mp3 renamed as electro_French79_AfterParty.mp3
French79_BetweentheButtons.mp3 renamed as electro_French79_BetweentheButtons.mp3

$ ls -l
total 2
-rw-r--r-- 1 cas cas 0 Oct 21 14:30 electro_DaftPunk_VeridisQuo.mp3
-rw-r--r-- 1 cas cas 0 Oct 21 14:30 electro_French79_AfterParty.mp3
-rw-r--r-- 1 cas cas 0 Oct 21 14:30 electro_French79_BetweentheButtons.mp3
1

If, despite of warnings given here already, you insist on a solution using the output of ls then the pipe is your friend.

I used double quotes to avoid problems from spaces in filenames, and I used braces to allow using additional text after the variable, if such should become needed.

ls -a1 *.mp3 |
while read fn; do 
mv "$fn" "electro_${fn}"
done

There is problems, not mentioned in other warnings, with using the output of ls.

In applying the *.mp3 glob, the -a switch is being ignored. (Globbing is designed to ignore hidden files.) Therefore, if you have any hidden .mp3 files in the directory, they will not be processed.

If, by random chance, you also have a directory, or more than one, with a name ending .mp3, ALL the files in that directory will be listed by ls, including any which are not .mp3 files. Those files will not be processed, but you will get an error for each one of them, including the . and .. entries, as well as the directory itself (because a : is appended to the name in the output) and for the blank line which precedes each sub directory listed.

The solution for those problems is to drop the glob and add a filter.
This is BASH specific, and might not work in other shells. (YMMV)

ls -a1 | grep '\.mp3$' |
while read fn; do
[[ $fn =~ ^([.]?)(.*)\.mp3 ]]
mv "$fn" "${BASH_REMATCH[1]}electro_${BASH_REMATCH[2]}.mp3"
done

That will find the hidden files, and keep them hidden. It will not attempt to traverse into any sub directories, no matter how named. By expanding the regular expression, it will also allow for some extra considerations in how you rename the files.


Explanation of the commands above:

ls -a1 : List all the entries in the current directory. The -a (all) is to find all files, including hidden ones. It could also be -A (almost all) which skips the . and .. entries while showing other hidden entries. I stayed with the option used by the OP. The 1, which is really -1 is to place each entry on its own line.

grep '\.mp3$': Use the regular expression to search for, and only output lines which end in .mp3. The dot normally means "any character" in regular expressions, so it has to be escaped with the backslash \. To prevent the Bash shell from trying to process and expand the pattern of the regular expression, it is enclosed in single quotes.

The output of the ls command is piped | to the input of the grep command, which has its output piped to the read command's input.

Since the read command is not given a source, it reads from the standard input, normally the terminal. In this case standard input is filled with the output from the grep command. Once there is nothing left to read, the read command will return 'false' (in shell-speak) and the while will exit. Each line found by the read command is assigned, in turn, to the fn variable for processing in the loop.

[[ $fn =~ ^([.]?)(.*)\.mp3 ]]: Most of the magic happens right here. The [[ .. ]] construct is not as portable as the [ .. ] construct. (I don't know about its POSIX conformance either.) Some, but not all, shells have it. Bash does, and it has some special properties which are especially helpful in this case. First off, it allows the use of unquoted variables on the left side of the operator while not splitting on spaces as Bash normally does. Rather convenient when dealing with filenames which can have all manner of surprises for scripts.

The other property of importance here is that it allows the use of the =~ operator for regular expression matching, rather than shell-style pattern matching, which the [ ..] construct is limited to. I opted for regular expression as the tool for processing the filenames and keeping the leading-dot hidden files as hidden while still renaming them. While the same thing can be done in a POSIX conformant manner and with shell pattern matching, it would not be as simple to write, or to modify.

As noted already in comments, the [[ .. ]] construct is normally used as a test, returning success or failure of the test inside. In this case, however, there is no need to check the return value as the inputs to the loop have already been filtered by grep and it will always succeed. The whole could be re-coded without the grep and using the return code here to process, or not, the file. I cannot say which is better from a performance stand point, and I suspect it's marginal at best either way.

Decomposing the regular expression goes like this. The ^ anchors the test pattern to the beginning of the string, so that nothing is skipped. Each of the ( .. ) pairs forms a "capturing group", which we'll get to in a moment. The [.] forms a "character class" composed of just the dot. This is another way to override the normal for regular expressions where the dot is a place holder for "any character", as the backslash was above, and will be again. The ? means to accept one or none of what is before it. So ([.]?) will "find" a single dot or nothing, and whichever that is will be remembered in the first capture group. The .* means to find any character . as many times as possible *. The common term for that is greedy, in that it will take as many characters as it can, but will not take so many as to force the rest of the pattern to fail. Again, since it is in parenthesis, (.*) will find as much as it can, and remember it in the second capture group. The final part, \.mp3 will again match a literal dot followed by "mp3". That could have been written as [.]mp3 as well. Since this section is not followed by any quantity qualifiers, the ? and * before, it must match or the whole thing fails. The greedy operation of the * will leave enough for this part to match. Had this also been marked with either qualifier, it would have been optional, and the second group could have taken everything, leaving nothing at the end.

When using regular expressions like this, Bash will place the captured groups into an array, BASH_REMATCH. The element of BASH_REMATCH with index 0 is the portion of the string which matches the pattern. In this case that is the entire filename. The element of BASH_REMATCH with index n is the portion of the string matching the nth parenthesized subexpression. In this case that is, 1: for the dot, or not, and 2: for the rest of the name, without the .mp3.

The mv command uses this array to rebuild the new filename. ${BASH_REMATCH[1]} is the first, possibly empty, capture group. ${BASH_REMATCH[2]} is the rest of the name, without the extension. So
mv "$fn" "${BASH_REMATCH[1]}electro_${BASH_REMATCH[2]}.mp3"
will rename the files without taking the leading dot and burying it inside the new name. A simple mv "$fn" "electro_${fn}" as I used in the first snippet, would rename the hidden file .track_2.mp3 to electro_.track_2.mp3. Not a good result. This way .track_2.mp3 becomes .electro_track_2.mp3.


The primary reason for using regular expression over Bash pattern matching here is for the option to expand how things are processed later. By way of example, playing with the sample names given, and adding more to demonstrate, it is possible to rearrange the names, and do several things with them. Also added is the ability to skip over directories in the list. Using this snippet, (which includes nested capture groups to show how they are counted):

ls -A1 | while read fn; do
if [ ! -d $fn ]; then
if [[ $fn =~ ^([.]?)(([^_]*)(_))?(.*)[.]mp3$ ]]; then
mv "${BASH_REMATCH[0]}" "${BASH_REMATCH[1]}electro_${BASH_REMATCH[5]}${BASH_REMATCH[2]:+${BASH_REMATCH[4]}by_${BASH_REMATCH[3]}}.mp3"
fi
if [[ $fn =~ ^([.]?)(([^_]*)(_))?(.*)[.]flac$ ]]; then
mv "${BASH_REMATCH[0]}" "${BASH_REMATCH[1]}electro-${BASH_REMATCH[5]}${BASH_REMATCH[2]:+ (by ${BASH_REMATCH[3]})}.flac"
fi
fi
done

results in this:

gypsy@suse-office:/se/unix_linux> ls -A1
DaftPunk_VeridisQuo.flac
DaftPunk_VeridisQuo.mp3
DaftPunk_VeridisQuo.wav
extras_tracks.mp3
French79_BetweentheButtons.flac
French79_BetweentheButtons.mp3
French79_BetweentheButtons.ogg
French79_Hometown.flac
French79_Hometown.mp3
.PrivateStock_OnTheBottle.flac
.PrivateStock_OnTheBottle.mp3
Track03.mp3
track_surplus.flac
gypsy@suse-office:/se/unix_linux> ls -A1 | while read fn; do
> if [ ! -d $fn ]; then
> if [[ $fn =~ ^([.]?)(([^_]*)(_))?(.*)[.]mp3$ ]]; then
> mv "${BASH_REMATCH[0]}" "${BASH_REMATCH[1]}electro_${BASH_REMATCH[5]}${BASH_REMATCH[2]:+${BASH_REMATCH[4]}by_${BASH_REMATCH[3]}}.mp3"
> fi
> if [[ $fn =~ ^([.]?)(([^_]*)(_))?(.*)[.]flac$ ]]; then
> mv "${BASH_REMATCH[0]}" "${BASH_REMATCH[1]}electro-${BASH_REMATCH[5]}${BASH_REMATCH[2]:+ (by ${BASH_REMATCH[3]})}.flac"
> fi
> fi
> done
gypsy@suse-office:/se/unix_linux> ls -A1
DaftPunk_VeridisQuo.wav
electro-BetweentheButtons (by French79).flac
electro_BetweentheButtons_by_French79.mp3
electro-Hometown (by French79).flac
electro_Hometown_by_French79.mp3
.electro-OnTheBottle (by PrivateStock).flac
.electro_OnTheBottle_by_PrivateStock.mp3
electro_Track03.mp3
electro-VeridisQuo (by DaftPunk).flac
electro_VeridisQuo_by_DaftPunk.mp3
extras_tracks.mp3
French79_BetweentheButtons.ogg
track_surplus.flac
gypsy@suse-office:/se/unix_linux> 
3
  • Shouldn't there be a binary operator after ]]? – Erwann Oct 21 '19 at 20:40
  • @Erwann Typical usage, yes; required, no. In this case [[ is used to gain simple access to regular expression power. – Chindraba Oct 21 '19 at 21:40
  • What is [[ $fn =~ ^([.]?)(.*)\.mp3 ]] for? – Erwann Oct 22 '19 at 1:36
0

Use IFS to set the delimiter. By breaking on newlines, whitespace is not an issue.

IFS=$(echo -en "\n\b")
for x in $(ls -1 *.mp3); do mv "$x" "electro_${x}"; done

Naturally, you might ask why not simply

IFS='\n'

This is because $() or backticks is command substitution, which drops trailing newlines.

From the BASH manual

\n newline

\b backspace

My understanding is this "goes back" (move left) to prevent stripping the trailing newline character, which allows it to be iterated on in the for (each) loop.

This is discussed in more detail in several other stackoverflow questions

2
  • 1
    "command substitution ... drops trailing newlines" this part is correct, but the rest isn't. The \b helps keep the \n because then the newline isn't a trailing newline, it now has one more character after it. – muru Oct 21 '19 at 5:22
  • 1
    why? for x in *.mp3 is better and shorter, and works with any valid filename. parrsing ls doesn't, no matter how you set IFS. Also, you don't need the echo -en"\n\b" trick, you can just use IFS=$'\n' - that still won't make parsing ls a good idea, but at least it's not a work-around for a self-made problem. – cas Oct 21 '19 at 9:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.