I have a couple of files in a Unix folder, let's say /home/TRANS.

Files are received within this folder on a monthly basis. File names are like:

  • ENCD_213_E-DM_CCA_ID3490_A01.txt
  • ENCD_213_E-DM_CCA_ID33120_A01.txt
  • ENCD_213_E-DM_CCA_IDP3664_A01.txt
  • ENCD_213_E-DM_CCA_ID3327_A01.txt
  • ENACT_215_E_DM_CCA_IDA33320_25OCT2017.csv
  • ENACT_215_E_DM_CCA_IDA31116_25OCT2017.csv

After renaming, the final output should be:

  • id3490.txt
  • id33120.txt
  • idp3664.txt
  • id3327.txt
  • ida33320.csv
  • ida31116.csv

So essentially, I want to have the following strings replaced across all file names within the TRANS folder, and make the final output filename as lowercase:

  • ENCD_213_E-DM_CCA_
  • _A01
  • ENACT_215_E_DM_CCA_
  • _25OCT2017

How can I execute this either in a single line command or as a shell script? I did check multiple questions but couldn't find replacement of multiple characters in addition to changing to lowercase. Need your help as I am pretty new to Unix.

  • are this parts _213_ and _215_ always fixed (the same) for all filenames? – RomanPerekhrest Oct 25 '17 at 15:12
  • That right Roman, _215_E_DM is always associated with the csv filesnames with date. – NiCKz Oct 25 '17 at 15:23
  • @NiCKz Hold on a sec and I'll add an explanation to my solution. – igal Oct 25 '17 at 15:24

There are a lot of ways to do this. Here is one command that should work for your example:

for oldname in *; do
    newname="$(echo "${oldname}" | grep -Po 'ID\w?\d+' | tr A-Z a-z).txt" \
    && mv -i "${oldname}" "${newname}";

That example used Perl-compatible regular-expressions which it appears your grep does not support. Here is an alternative using basic regular expressions:

for oldname in *; do
    newname="$(echo "${oldname}" | grep -o 'ID[A-Z]\?[0-9]\+' | tr A-Z a-z).txt" \
    && mv -i "${oldname}" "${newname}";

A more detailed explanation follows.

The expression for oldname in * uses a globbing-pattern/wildcard to iterate over the files in your current directory and stores each name in the oldname variable. To test this you could run the following:

for oldname in *; do echo "${oldname}"; done

Then we use grep to extract the part of the file-name you want to keep. The -P option flag tells grep to use Perl-compatible regular expressions (not really necessary in this situation) and the -o flag tells grep to extract only the matching substring (instead of printing the entire string). The \w? pattern matches an option single character (a "word" character) and the \d+ pattern matches one or more digits. We can test out the regular expression like this:

for oldname in *; do echo "${oldname}" | grep -Po 'ID\w?\d+'; done

Then we use tr to convert uppercase characters to lowercase:

for oldname in *; do echo "${oldname}" | grep -Po 'ID\w?\d+' | tr A-Z a-z; done

The next step is to use command substitution to assign this string to a variable and then print the result:

for oldname in *; do
    newname="$(echo "${oldname}" | grep -Po 'ID\w?\d+' | tr A-Z a-z)" && echo "${newname}"

Then we add in the ".txt" file-extension:

for oldname in *; do
    newname="$(echo "${oldname}" | grep -Po 'ID\w?\d+' | tr A-Z a-z).txt" && echo "${newname}"

We can run this command as a sanity-check to make sure we're getting the expected results. Once we're satisfied we replace the echo command with a mv command:

for oldname in *; do
    newname="$(echo "${oldname}" | grep -Po 'ID\w?\d+' | tr A-Z a-z).txt" \
    && mv -i "${oldname}" "${newname}";
  • Thanks @igal. Can this be executed in the same command line? Would you mind adding an explanation of what each option does? Sorry, but I'm completely new and wanted to understand. A link may also help. – NiCKz Oct 25 '17 at 15:25
  • Also, is there a Preview option available for this command so that I can check before finally executing it to test in my folder? I saw in some of the posts that a -n option works as a preview, but not sure if it would with this one. – NiCKz Oct 25 '17 at 15:29
  • @NiCKz Check out my updated answer. I think it will address your concern. Let me know if it doesn't. – igal Oct 25 '17 at 15:35
  • @NiCKz There's no general "dry run" flag that will work with every command. Some commands (e.g. rsync) have options like that, but it isn't typical. – igal Oct 25 '17 at 15:46
  • Thanks for the detailed explanation.. very helpful. However, when I am executing this: for oldname in *; do echo "${oldname}" | grep -Po 'ID\w?\d+'; done I get this error message: grep: illegal option -- P grep: illegal option -- o Also, I have some csv files that would need to be renamed, which has dates in them. – NiCKz Oct 25 '17 at 15:52

find + bash solution:

find . -type f -regextype posix-egrep \
    -regex ".*EN(ACT|CD)_[0-9]+_E(-|_)DM_CCA_.+[0-9]\.(txt|csv)$" -exec bash -c \
    'fn=${0##*/}; dir_n="${0%/*}/"; 
    [[ "$fn" =~ .*_(ID[^_]+)_.*\.(txt|csv)$ ]]; 
    mv "$0" "$dir_n${BASH_REMATCH[1],,}.${BASH_REMATCH[2]}"; ' {} \;

For filename manipulation, bash parameter expansions are very nice. See this for a good overview.

Since you're basically wanting to preserve the ID part of the file, you can do this:


for f in *csv *txt; do

   if [[ $f =~ ID[[:alnum:]]+ ]]; then
      mv "$f" "${BASH_REMATCH,,}.${ext}"



This loops over every csv and txt file and gets the extension using PE. Then, use the bash regex match operator =~ to check if the filename matches your pattern. If it does, bash populates $BASH_REMATCH with the contents of the regex match. Then, move the file to the lowercase version of this match, appending the original extension. I created all of your example files and got the intended results.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.