2

I have thousands of text files inside a folder and they are named in a certain way.

For example:

Image_234_Data_7778_n0.txt
Image_234_Data_7778_n1.txt
Image_234_Data_7778_n2.txt

Image_954_Data_4478_n0.txt
Image_954_Data_4478_n1.txt
Image_954_Data_4478_n2.txt

Image_104_Data_9878_n0.txt
Image_104_Data_9878_n1.txt
Image_104_Data_9878_n2.txt

And so on...

I would like to make a sourcefile (which would be used to rename the files and keep track of 'which is which') that should read:

1_1.txt:Image_234_Data_7778_n0.txt
1_2.txt:Image_234_Data_7778_n1.txt
1_3.txt:Image_234_Data_7778_n2.txt

2_1.txt:Image_954_Data_4478_n0.txt
2_2.txt:Image_954_Data_4478_n1.txt
2_3.txt:Image_954_Data_4478_n2.txt

3_1.txt:Image_104_Data_9878_n0.txt
3_2.txt:Image_104_Data_9878_n1.txt
3_3.txt:Image_104_Data_9878_n2.txt

And so on...

Can anyone help me with some code which could do this for me?

  • 2
    Please edit your question and explain how you want them to be renamed. How can we decide which file should be 1_1.txt or 3_3.txt? Presumably, you want to parse the file name but you need to explain exactly how. How would this "sourcefile" be used to rename the files? Do you want to create the sourcefile only or do you also want to rename the files? – terdon Aug 11 '14 at 13:42
  • @terdon:I don't care about the order but after the files are renamed, I need to have a logfile which keeps track of 'who is who'. Because I will need to go back and be able to relate the old filenames and new filenames. Thank you ! – Tofayel Aug 28 '14 at 4:16
  • @user3810095 check this command rename -v 'our $i+=1;s/(.*)_n(\d+)\.txt$/$i\_$2.txt/' *.txt > which_is_which. this command rename you files like 1_0.txt ,2_1.txt, 3_2.txt ... 10_0.txt, 11_1.txt, 12_2.txt and so on. And at the end which_is_which file contain which files renamed to which. – αғsнιη Dec 5 '14 at 16:51
1

I assume that your file names are of the form XXXXNNN.txt where XXXX is some arbitrary text not ending in a digit and NNN is a sequence of digits, and that you want to group them by groups of identical XXX.

Strategy: loop over the files in lexicographic order, and detect when the XXXX part changes. For each group, generate the new names. A small complication is that if the NNN parts are variable-width, then they are not sorted lexicographically: NNN=10 will appear between NNN=1 and NNN=2.

current=
numbers=
i=0
for x in *.txt ''; do
  stem=${x%.*}
  n=${stem##*[!0-9]}
  stem=${stem%$n}
  if [ "$stem" != "$current" ]; then
    for k in $(printf '%s\n' $numbers | sort -n); do
      y=${i}_${k}.txt
      echo mv "$current$k.txt" "$y"
    done
    current=$stem
    numbers=$n
    i=$((i+1))
  else
    numbers="$numbers $n"
  fi
done

Replace echo mv by the command you want to use, e.g. mv to rename the files or echo … to write information to a file.

Instead of renaming files and keeping track of the old names, consider using symbolic links so that the files are accessible both as their original names and with the simplified names.

  • Thanks a ton Gilles for your answer...I apologise for being very late in acknowledging it...it works fine for all of my cases....but is there any way to keep track of who gets converted to which name. I mean, it would be great if I could also have a logfile being generated which keeps track (oldfilename:newfilename)of the history of conversion. I have got few thousand files and its hard to go back and check 'who is who'. Thank you !!! – Tofayel Aug 28 '14 at 4:13
  • @user3810095 A simple method would be to use ln -s instead of mv. That way you would be able to access the files from either name. If you want to rename and keep track of the renamings, you can use both the echo mv and the mv line. – Gilles Aug 28 '14 at 7:55
  • Thank you for your suggestion. I had to make few modifications for my current need. Removed the '_' in y=${i}_${k}.txt line. Also replaced 'echo mv' with 'echo' and added "ln -s {$current$k.txt} {$y} after the echo line. Also added : '> sourcefile.txt' at the end to write the whole thing in a textfile. Now, only 1 job remaining for me and that is to add a zero padding before the filenames, which should be adaptable to make the length of the new filenames consisting of say 5 or 7 digits. So, 11.txt should become 00011.txt, 3452.txt should become 03452.txt. Any suggestions? – Tofayel Aug 31 '14 at 11:18
  • @user3810095 Start with i=100000 instead of i=0, and use ${i#1} instead of $i to cut out the leading 1. Alternatively, use printf %05d somewhere. – Gilles Aug 31 '14 at 19:30
0

After creating a test directory containing the following files:

Image_104_Data_9878_n0.txt, Image_104_Data_9878_n1.txt,
Image_234_Data_7778_n0.txt, Image_234_Data_7778_n1.txt,
Image_234_Data_7778_n2.txt, Image_954_Data_4478_n0.txt,
Image_954_Data_4478_n1.txt, Image_954_Data_4478_n2.txt

I then did:

printf %s\\n * | sort --debug -t_ -k2,2n -k5.2n,5.2n 

And the results were:

Image_104_Data_9878_n0.txt
      ___
                     _
__________________________
Image_104_Data_9878_n1.txt
      ___
                     _
__________________________
Image_234_Data_7778_n0.txt
      ___
                     _
__________________________
Image_234_Data_7778_n1.txt
      ___
                     _
__________________________
Image_234_Data_7778_n2.txt
      ___
                     _
__________________________
Image_954_Data_4478_n0.txt
      ___
                     _
__________________________
Image_954_Data_4478_n1.txt
      ___
                     _
__________________________
Image_954_Data_4478_n2.txt
      ___
                     _
__________________________

I told sort to sort primarily by number on the 2cd _ delimited field from the beginning of the field to the end of the field like -k2,2n and secondarily to sort numerically on the second 2cd byte in 5th field like -k5.2,5.2n. I asked it for --debug output so it would show me just what the hell it was doing.

I could have sorted primarily on the 4th field as easily, or primarily on field 2, secondarily on field 5.2, and least important by the second field. I say all of this because I cannot determine any rhyme or reason to the sort offered in your example and I can only assume you assigned them like:

  • 1_1 : 234/7778
  • 2_1 : 954/4478
  • 3_1 : 104/9878

...because you, as yet, do not have any proper command worked out to sort them and perhaps you need a little advice on how you might do so. Following on with that assumption, I will do this:

printf %s\\n * | 
sort -t_ -k4,4n -k5.2n,5.2n | 
nl -bp'_n0\.' -s_ |
sed 's/\(I[^.]*_n\)\(.*\)/\2:\1\2/;N
     s/ *\([0-9]*_\)\(.*\n\) *\([^_]*I\)/\1\2\1\3/;P;D'

That produces results that I believe should be pretty close to what you're looking for. See?

1_0.txt:Image_954_Data_4478_n0.txt
1_1.txt:Image_954_Data_4478_n1.txt
1_2.txt:Image_954_Data_4478_n2.txt
2_0.txt:Image_234_Data_7778_n0.txt
2_1.txt:Image_234_Data_7778_n1.txt
2_2.txt:Image_234_Data_7778_n2.txt
3_0.txt:Image_104_Data_9878_n0.txt
3_1.txt:Image_104_Data_9878_n1.txt

There they get sorted and numbered by the 4th field because I specified the -k4,4n field to sort but you could as easily do -k2,2n as noted.

The command works by asking nl to number only lines containing the string _n0.. sed receives its output which looks like:

 1_Image_954_Data_4478_n0.txt
   Image_954_Data_4478_n1.txt
   Image_954_Data_4478_n2.txt
 2_Image_234_Data_7778_n0.txt
   Image_234_Data_7778_n1.txt
   Image_234_Data_7778_n2.txt
 3_Image_104_Data_9878_n0.txt
   Image_104_Data_9878_n1.txt

...and first copies the _n[0-9]*.txt bit to the head of the line, Next pulls in the next line, and, if the pattern space looks like:

 *num_.*\n [^_]*I

...at that point it tacks the num bit from the first line onto the second. If you wanted to go from the text file that command would produce to a move operation you could do:

sed 's/\([^:]*\):\(.*\)/$* \2 \1/' <txtfile |
sh -s -- echo mv

OUTPUT

mv Image_954_Data_4478_n0.txt 1_0.txt
mv Image_954_Data_4478_n1.txt 1_1.txt
mv Image_954_Data_4478_n2.txt 1_2.txt
mv Image_234_Data_7778_n0.txt 2_0.txt
mv Image_234_Data_7778_n1.txt 2_1.txt
mv Image_234_Data_7778_n2.txt 2_2.txt
mv Image_104_Data_9878_n0.txt 3_0.txt
mv Image_104_Data_9878_n1.txt 3_1.txt

There it is only echoed because that is the shell process's first parameter, but if you remove it, and run it, as I have just done, you'll like get the same results:

ls -m

1_0.txt, 1_1.txt, 1_2.txt, 2_0.txt, 2_1.txt, 2_2.txt, 3_0.txt, 3_1.txt

Gilles recommends links, which I also think is an excellent idea, though I would personally steer clear of softlinks if I could help it and just do a mirrored hardlink directory. You could do that in almost the exact same way, but you'd want to use ln as opposed to mv.

  • Thank you very much for the answer Mike. I also apologise for being very late in my acknowledgement.I could use your code without any problem for the text files but couldn't extend it to my other cases. Given I am a complete novice in the scripting field (majored in Biochemistry), I could not extend it to my other set of files, having names that are a bit different. Could you please take some time to suggest me what should I do when my filenames are in this format: Text_23335981_Data_22317866_22317867_20140723_0935_n0.Thank you !!! – Tofayel Aug 28 '14 at 4:32
  • @user3810095 - I think it's just a matter of switching out the T for an I but I have to check. Still - I don't know which of those number fields you wish to sort on or why... – mikeserv Aug 28 '14 at 7:46
  • My bad ! It works fine now....could you please suggest me some modification in your code to add a zero padding at the beginning of the new filenames? I mean, if I want something like this- 00001_0.txt, 00001_1.txt, 00001_2.txt and so on. But I would like it to be consisting of 5 digits before the '_' (So, the padding is adaptable).Also, if I end up obtaining the new filename 00266_2.txt inside a directory, I would like to start renaming another fresh batch of files (in a different directory) which should start from 00267_0.txt and do the same job all over again. Thanks a ton for all the help – Tofayel Aug 29 '14 at 14:54
  • To answer your Q: I dont have any preferred order of arrangement for the files. Just that, I have thousands of files that are generated by one of my machine and they come in a set of three files. Thats where you get the names n0,n1,n2 at the end of filename. SO keeping the sets of n0,n1,n2 together is important but I don't care about which set gets the priority while renaming. Thank you ! – Tofayel Aug 29 '14 at 15:07
  • @user3810095 You can get the zero-padding with adding two option for nl, I believe. No, I know - I just tested it, You specify the width with -w and the padding with -nrz. So if you change the nl line to nl -bp'_n0\.' -s_ -w5 -nrz | you'll get output like 00001_0.txt:Image_954_Data_4478_n0.txt and so on. – mikeserv Aug 30 '14 at 2:40
0

One trick for things like this – particularly if you're not too handy with scripting – is to generate a script with a spreadsheet. This isn't good practice if you're trying to build a script for repeated use, but it can be handy for a one-off job for someone who hasn't had the time to learn to do scripting another way.

It looks like the rename (or link) you're trying to accomplish is to change everything before the _n into a single sequence number before a _, and leave the portion after the _n unchanged. If that's not your exact intention, it's easy enough to revise a spreadsheet formula.

For example (shortening your file names to reduce the need to scroll sideways):

A2               B2             C2           D2           E2
i234d7778_n0.txt =FIND("_n",A2) =LEFT(A2,B2) =D1+(C2<>C1) =D2&"_"&RIGHT(A2,LEN(A2)-B2-1)
i234d7778_n0.txt 10             i234d7778_   1            1_0.txt

The reason this starts on row 2 (not 1) is so that the comparison in D2 goes to row 1. (An alternative is to put it on row one, but special-case D1 to 1.)

The expression in D2 is just a terse way of saying this: =IF(C2=C1,D1,D1+1)

As normal with spreadsheets, paste your list of files into column A, and repeat columns B through E as many times as there are files. To generate a script to do the rename, you can add an "F" or "G" column:

F2                          G2
="mv "&A2&" "&E2            ="ln "&A2&" new-name-directory/"&E2
mv i234d7778_n0.tx 1_0.txt  ln i234d7778_n0.tx new-name-directory/1_0.txt

The F and G columns provide the text for an executable script.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.