0

I have a directory tree with a few thousand files collected over the years with random and useless file names.

I want to clean them up, keep them relevant, remove certain words but also not create any duplicate filenames.

For example

WONDERBROS - PAWG Remy LaCroix Gets BREAKFAST! (pwg11717) - SOMENAME.mp4

I want that one to be renamed something like

WONDERBROS.PAWG.Remy.LaCroix.Gets.BREAKFAST!.(pwg11717).mp4

So a few points are

  • replace all spaces with .
  • remove SOMENAME whenever it occurs
  • Check that this filename does not already exist before renaming and if it does, append a number and keep trying until it is a unique filename so that nothing is overwritten

I think that this can be done with a shell script but if anyone has a tool to recommend I would like to try that too.

I tried something like this and only the first line seems to work.

# find and replace spaces with .
find /home/matt/rename_test_tmp/ -depth -name "* *" -execdir rename 's/ /./g' "{
# find and replace somename with .
find /home/matt/rename_test_tmp/ -depth -name "somename" -execdir rename 's/ 
# find and replace SOMENAME
find /home/matt/rename_test_tmp/ -depth -name "SOMENAME" -execdir rename 's/ 
# find and replace Somename
find /home/matt/rename_test_tmp/ -depth -name "Somename" -execdir rename 's/ 

I think my problem is that I still do not understand reg ex?

  • should filenames be unique across all subfolders or in regard to the common folder? – RomanPerekhrest Feb 24 '18 at 21:38
  • none of your example find commands are complete, so none of them will run. btw, which version of rename do you have - the perl version of rename, or the one from util-linux? they are very different and not compatible with each other. Run rename -V if you're not sure. – cas Feb 24 '18 at 23:54
  • @RomanPerekhrest, ideally, yes i think unique across subfolders but I can cope with it if there are conflicts with files in other directories. – Matt_the_invader Feb 25 '18 at 10:03
  • @cas Rename version 0.20 from the ubuntu repository (i´m using linux mint). Odd, the first line seems to work, It got rid of all the spaces and replaced them with . I am happy to try other things though, I don´t really know what Iḿ doing to be honest so am looking forward to learning better way´s from more knowledgeable people. – Matt_the_invader Feb 25 '18 at 10:06
0

Here is a basic structure. If something goes wrong, MAXTRYS stops the renaming. Chose a reasonable value.

#!/bin/bash
#
# Usage: 
#       numberedMove xy-file.txt TARGETDIR 
#
file="$1"
targetDir="$2"
MAXTRYS=666
#
# @TODO
# add tests for permissions, maybe handle symlinks etc.
#
test -f "$file" || {
    echo No such file "$file" or not an ordinary file
    exit 1;
}

test -d "$targetDir" || {
    echo No such directory "$targetDir"
    exit 2;
}

#
# append NUM+1 to filename, to create unique name
#
numbered () {
  fname="$1"
  num=$2
  if [[ $num -gt $MAXTRYS ]]
  then
    echo " giving up - max trys: $MAXTRYS "
    exit 3
  fi
  # echo "mv $fname → $targetDir/$fname$num"
  test -f "$targetDir/$fname$num" && numbered "$fname" $((num+1)) || mv "$file" "$targetDir/$fname$num"
}

#
# remove all SOMENAME and replace every blank with a dot
#
filtername () {
   fname="$1"
   fname=${fname//SOMENAME/}
   fname=${fname// /.}
   echo "$fname"
}
#
# filter filename, if unique: move, else move with number suffix
#
if [[ -e "$file" ]]
then
    newname=$(filtername "$file")
    if [[ ! -f "$targetDir/$newname" ]]
    then
        # echo "mv $file → $targetDir/$newname"
        mv "$file" "$targetDir/$newname"
    else
        # echo "mv $file → $targetDir/$newname NUMBERED"
        numbered "$newname" 1
    fi
fi

In filtername you may add more rules and change the existing ones.

If this was for my own needs, I would replace blanks with '-', not '.', since the dot is a, fair enough, weak indicator (a.tar.bz2) for meaning.

And I would collapse multiple delimiter to one, so instead of:

    mv "ab cd - SOMENAME..mp3" → "./B/ab.cd.-...mp3" 
    mv "ab cd - SOMENAME..mp3" → "./B/ab-cd-.mp3" 

And I would keep the file extension, and put the number in front of the last extension:

    mv "ab cd- SOMENAME..mp3"  → "./B/ab-cd-0.mp3" 
    mv "ab cd - SOMENAME..mp3" → "./B/ab-cd-1.mp3" 
    mv "ab cd -SOMENAME..mp3"  → "./B/ab-cd-2.mp3" 

since many programs interpret the extension.

I only tested the program with about 10 files, so make a backup before using, and check, maybe by total file size, and file count, whether the operation succeeded.

My testing:

for f in a* ; do ./numberedMove.sh "$f" ./B ; done

Note, that when placing the program in the CWD, it might get moved itself.

  • This is interesting, I´m picking through it now trying to better understand it as it does not run as it is. sh filtername.sh No such file or not an ordinary file – Matt_the_invader Feb 25 '18 at 10:35
0

If you're using the perl rename utility, try something like this:

find . -type f -execdir rename -n 's/\s*-\s*/./g; s/\s+/./g; s/somename//ig' {} +

The \s*-\s* matches a - surrounded by zero-or-more whitespace characters (i.e. a - with optional whitespace). \s+ matches one-or-more whitespace characters.

You can add as many s/search/replace/ commands in the rename script as you like. Note, though, that you need to be careful about the order of execution - e.g. if you want to change "foo" to "bar" and "food" to "drink", then you need s/food/drink/ before s/foo/bar/ because the latter will change food to bard.

The command above is applied to ALL filenames. If some of the s/search/replace/ commands do not apply to a particular filename, it's not an error - that particular command is not applied (but others still are). Even so, you may need to run multiple find commands using the -name or -iname predicate if you want some renames to only be applied to certain file names.

You can also use alternation if you want to change multiple words to the same replacement. e.g.

s/(somename|someothername|thisname|thatname)//ig

that changes all of them to the empty string (i.e. removes them). The /i modifier makes it case-insensitive.

The -n option from rename is a dry-run option - it will only show what would be renamed, without actually renaming anything. Remove the -n from the rename command when you've verified that it does what you want, and nothing that you don't want. If you want verbose output while renaming, replace the -n with -v.

from man rename:

-v, -verbose

Verbose: print names of files successfully renamed.

-n, -nono

No action: print names of files to be renamed, but don't rename.


Note: The perl rename utility will not rename a file if the new filename already exists. It has no built-in ability to add numbers to filenames in case of collision, but one of the best features of this rename is that you're not restricted to just simple operations like s/search/replace/ - you can run any perl code inside a rename script and it will rename the source filename to whatever $_ is changed to (operators like s/// or y// implicitly modify $_ if an explicit variable name is not used. If you're new to perl, there's a pretty good explanation of what $_ is at https://perlmaven.com/the-default-variable-of-perl).

That allows something like this (untested, but might work)

find . -type f -execdir rename -n '
    s/\s*-\s*/./g; 
    s/\s+/./g;
    s/somename//ig;

    if (-f $_) {
      my $num='001';
      while (-f "$_.$num") {
        $num=sprintf('%03i',++$num);
      };
      $_ = "$_.$num";
    }' {} +

That should add a zero-padded 3-digit-wide number (i.e. from 001 to 999) to the filename if it already exists. Change %03i to %02i in the sprintf if you want only two digits (01 to 99). Or, if you don't expect any numbering of duplicate files to exceed 9, change it to just my $num=1, and then just $num++ inside the while loop with no sprintf().

with a little more work (by splitting the filename into "basename" and "extension" portions) the numbering could be inserted before the extension, rather than just appended to the end.


You can find out which version of rename you have with the -V option. e.g. on my Debian system rename is the perl rename while rename.ul is the util-linux rename:

$ rename -V
/usr/bin/rename using File::Rename version 0.20

$ rename.ul -V
rename.ul from util-linux 2.31.1
0
find . -maxdepth 1 -type f > orginal_file

find . -maxdepth 1 -type f -exec sed -r "s/\s+//g"| sed "s/SOMENAME//g" >> /var/tmp/modified file

paste orginal_file /var/tmp/modified file >> combined_orginal_modified_column_wise

for i in `cat /var/tmp/modified file`
do
if [[ -f $i ]]
then
echo "file exsist"
else
awk -v i="$i" '/i/{print "mv" " " $1 " " $2}'   combined_orginal_modified_column_wise
fi
done

for i in `cat /var/tmp/modified file`
do
if [[ -f $i ]]
then
for j in {1..10}
do
elif [[ -f $i$j ]]
then
echo "$i$j exsists"
fi
done
else
awk -v i="$i" -v j="$j" '{print "mv" " " $1 " " ij}'  combined_orginal_modified_column_wise
fi


done

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.