0

I want to parse data of particular columns from several files(aprilPlate.txt, mayPlate.txt, junePlate.txt, julyPlate.txt, augustPlate.txt) using For loop.

The data of input files (aprilPlate.txt, mayPlate.txt, junePlate.txt, julyPlate.txt, augustPlate.txt) look like:

Incl    Cal Ps  Name    Q   Con Std Status
True    255 A1  Sample 1    35.86       0   
True    255 A2  Sample 2    36.06       0   
True    255 A3  Sample 3    17.45       0   
True    255 A4  Sample 4    17.56       0   
True    255 A5  Sample 5    17.55       0   
True    255 A6  Sample 6    40.00       0   
True    255 A7  Sample 7    36.38       0   
True    255 A8  Sample 8    27.98       0   
True    255 A9  Sample 9    27.95       0   
True    255 A10 Sample 10   28.19       0   
True    255 A11 Sample 11   36.93       0   
True    255 A12 Sample 12   37.74       0   
True    255 A13 Sample 13   17.88       0   
True    255 A14 Sample 14   17.82       0   
True    255 A15 Sample 15   17.90       0   
.
.
.

By using FOR loop, I able to get the task done. However, in order to get desire data from all files, I have to modify the file name in the script (below) manually.

#!/bin/bash

# parse the data of desire columns from target file
# rename the column name 
# redirect the stdoutput to a text file

for z in A B; 
  do for i in 3 4 5 13 14 15; 
    do grep $z$i aprilPlate.txt | 
       awk -F "\t" '{print $3 "\t" $5}' | 
       sed -e 's/A[3-5]/st_SWC/g;s/A[1][0-9]/st_SWD/g;s/B[3-5]/st_TZC/g;s/B[1][0-9]/st_TZD/g;' >> stone.txt; 
  done;
done

for z in E F; 
  do for i in 8 9 10 18 19 20;
    do grep $z$i aprilPlate.txt | 
       awk -F "\t" '{print $3 "\t" $5}' | 
       sed -e 's/E[8-9]\|E[1][0]/su_SWC/g;s/E[1][0-9]\|E[2][0]/su_SWD/g;s/F[8-9]\|F[1][0]/su_TZC/g;s/F[1][0-9]\|F[2][0]/su_TZD/g;' >> suy.txt;
  done;
done

paste -d'\t' stone.txt suy.txt >> aprilPlate.data.txt
  • My apologies, I made a mistake in my previous coding. The correction was done.

The output of parsed data file should looks like:

st_SWC  17.45   su_SWC  28.85
st_SWC  17.56   su_SWC  28.79
st_SWC  17.55   su_SWC  28.82
st_SWD  17.88   su_SWD  29.24
st_SWD  17.82   su_SWD  29.18
st_SWD  17.90   su_SWD  29.23
st_TZC  18.06   su_TZC  25.99
st_TZC  18.09   su_TZC  25.98
st_TZC  18.13   su_TZC  26.02
st_TZD  17.75   su_TZD  25.00
st_TZD  17.70   su_TZD  25.01
st_TZD  17.69   su_TZD  24.98

I would like to ask, is there anyway I can have the files as 3rd variable in the script? Other solutions are welcome.

0

Something like will help you

for file in aprilPlate.txt  mayPlate.txt  junePlate.txt  julyPlate.txt  augustPlate.txt;
do
for z in A B; 
  do for i in 3 4 5 13 14 15; 
    do grep $z$i $file | 
       awk -F "\t" '{print $3 "\t" $5}' | 
       sed -e 's/A[3-5]/SWC/g;s/A[1][0-9]/SWD/g;s/B[3-5]/TZC/g;s/B[1][0-9]/TZD/g;' >> stone.txt; 
  done;
done
<snip>
done
  • Of course w/o commas between filenames (my mistake, edited) – Romeo Ninov Apr 30 '15 at 10:10
0

Not sure exactly what you are trying to do since you have no input or output examples, but the following should work.

#!/bin/bash
awksrc='BEGIN{FS=OFS="\t"}
{ gsub(/A[345]|E[89]|E10/, "SWC");
  gsub(/A1[0-9]|E1[1-9]|E20/, "SWD");
  gsub(/B[345]|F[89]|F10/, "TZC");
  gsub(/B1[0-9]|F1[1-9]|F20/, "TZD");
}
/SW[CD]{print $3, $5 >"stone.txt"}
/TZ[CD]{print $3, $5 >"suy.txt"}'
for file in aprilPlate.txt mayPlate.txt junePlate.txt julyPlate.txt ...
do
    awk "$awkscr" $file >/dev/null
    paste -d'\t' stone.txt suy.txt >> ${file%.txt}.data.txt
done

The ${file%.txt} gets the base name, without the extension to be able to add the .data.txt extension.

This reduces the number of shell loops to just one, and processes each file once. The output of the awk script is unneeded since the files are getting written to inside it.

Again, without example input, it's hard to say if this is correct for your purposes.

  • I have the input and output examples included in my question now. Thanks for your suggestion. – KJ Lim Apr 30 '15 at 7:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.