Replacing multiple characters across multiple File names in a Unix folder

Question

I have a couple of files in a Unix folder, let's say /home/TRANS.

Files are received within this folder on a monthly basis. File names are like:

• ENCD_213_E-DM_CCA_ID3490_A01.txt
• ENCD_213_E-DM_CCA_ID33120_A01.txt
• ENCD_213_E-DM_CCA_IDP3664_A01.txt
• ENCD_213_E-DM_CCA_ID3327_A01.txt
• ENACT_215_E_DM_CCA_IDA33320_25OCT2017.csv
• ENACT_215_E_DM_CCA_IDA31116_25OCT2017.csv

After renaming, the final output should be:

• id3490.txt
• id33120.txt
• idp3664.txt
• id3327.txt
• ida33320.csv
• ida31116.csv

So essentially, I want to have the following strings replaced across all file names within the TRANS folder, and make the final output filename as lowercase:

• ENCD_213_E-DM_CCA_
• _A01
• ENACT_215_E_DM_CCA_
• _25OCT2017

How can I execute this either in a single line command or as a shell script? I did check multiple questions but couldn't find replacement of multiple characters in addition to changing to lowercase. Need your help as I am pretty new to Unix.

Answer 1

There are a lot of ways to do this. Here is one command that should work for your example:

for oldname in *; do
    newname="$(echo "${oldname}" | grep -Po 'ID\w?\d+' | tr A-Z a-z).txt" \
    && mv -i "${oldname}" "${newname}";
done

That example used Perl-compatible regular-expressions which it appears your grep does not support. Here is an alternative using basic regular expressions:

for oldname in *; do
    newname="$(echo "${oldname}" | grep -o 'ID[A-Z]\?[0-9]\+' | tr A-Z a-z).txt" \
    && mv -i "${oldname}" "${newname}";
done

A more detailed explanation follows.

---

The expression for oldname in * uses a globbing-pattern/wildcard to iterate over the files in your current directory and stores each name in the oldname variable. To test this you could run the following:

for oldname in *; do echo "${oldname}"; done

Then we use grep to extract the part of the file-name you want to keep. The -P option flag tells grep to use Perl-compatible regular expressions (not really necessary in this situation) and the -o flag tells grep to extract only the matching substring (instead of printing the entire string). The \w? pattern matches an option single character (a "word" character) and the \d+ pattern matches one or more digits. We can test out the regular expression like this:

for oldname in *; do echo "${oldname}" | grep -Po 'ID\w?\d+'; done

Then we use tr to convert uppercase characters to lowercase:

for oldname in *; do echo "${oldname}" | grep -Po 'ID\w?\d+' | tr A-Z a-z; done

The next step is to use command substitution to assign this string to a variable and then print the result:

for oldname in *; do
    newname="$(echo "${oldname}" | grep -Po 'ID\w?\d+' | tr A-Z a-z)" && echo "${newname}"
done

Then we add in the ".txt" file-extension:

for oldname in *; do
    newname="$(echo "${oldname}" | grep -Po 'ID\w?\d+' | tr A-Z a-z).txt" && echo "${newname}"
done

We can run this command as a sanity-check to make sure we're getting the expected results. Once we're satisfied we replace the echo command with a mv command:

for oldname in *; do
    newname="$(echo "${oldname}" | grep -Po 'ID\w?\d+' | tr A-Z a-z).txt" \
    && mv -i "${oldname}" "${newname}";
done

Answer 2

find + bash solution:

find . -type f -regextype posix-egrep \
    -regex ".*EN(ACT|CD)_[0-9]+_E(-|_)DM_CCA_.+[0-9]\.(txt|csv)$" -exec bash -c \
    'fn=${0##*/}; dir_n="${0%/*}/"; 
    [[ "$fn" =~ .*_(ID[^_]+)_.*\.(txt|csv)$ ]]; 
    mv "$0" "$dir_n${BASH_REMATCH[1],,}.${BASH_REMATCH[2]}"; ' {} \;

Answer 3

For filename manipulation, bash parameter expansions are very nice. See this for a good overview.

Since you're basically wanting to preserve the ID part of the file, you can do this:

#!/bin/bash

for f in *csv *txt; do
   ext="${f##*.}"

   if [[ $f =~ ID[[:alnum:]]+ ]]; then
      mv "$f" "${BASH_REMATCH,,}.${ext}"
   fi

done

exit

This loops over every csv and txt file and gets the extension using PE. Then, use the bash regex match operator =~ to check if the filename matches your pattern. If it does, bash populates $BASH_REMATCH with the contents of the regex match. Then, move the file to the lowercase version of this match, appending the original extension. I created all of your example files and got the intended results.