I have a data file of this format (top row is for reference):

    E          T               N         D                 S             s                 R          k
   0.45847  300.0000      9.99979156  0.44254427E+02  0.49221658E-04  0.98763353E+19  0.73239256E-09  0.24892539E+15
   0.45947  300.0000      9.97603847  0.51763106E+02 -0.24189016E-04  0.97909193E+19 -0.24467623E-08  0.23855274E+15

Now I need to generate a file in the same column format containing these:

N         (k/(s/300.0000))         Abs(S)/E         exp(Abs(S)/E)

In short, I first need to divide the two columns and take the absolute value of Column(5) and again perform the exponentiation.

I am using the code:

while read -a line; do echo -e " ${line[2]}\t" `awk "BEGIN {print ${line[7]}/${line[5]}/300}"` ; done < filename

But this only gets me to the 2nd column. CI need to take the absolute value as in the next two columns?

I have come till:

 while read -a line; do echo -e ${line[4]} | awk ' { if($line[4]>=0 ) {print $line[4] } else {print $line[4]*(-1)} }'; done <

But I can't combine this with the last column-wise division code. Am I missing some syntax with BEGIN or END usage in awk? Also the result comes out messed up in decimal format. A

P.S.: Is awk the right tool to do this? Shall I try python or something? I know I can't use spreadsheets as I will have over 2 dozens of these files.


Yes, awk is correct tool as you could do:

$ awk 'function abs(x) {return x<0 ? -x : x}
     NR>1{printf ("%e %e %e %e\n", $3, $NF/($6/300.0000), abs($5)/$1, exp(abs($5)/$1) );
}' infile

9.999792e+00 7.561268e-03 1.073607e-04 1.000107e+00
9.976038e+00 7.309408e-03 5.264547e-05 1.000053e+00

You can simulate abs() function as like above, or use a Ternary Operator to generate it like as following without defining a function:

$ awk 'NR>1{ $5*=($5<0?-1:1);
    printf("%e %e %e %e\n", $3, $NF/($6/300.0000), $5/$1, exp($5/$1) )}' infile
9.999792e+00 7.561268e-03 1.073607e-04 1.000107e+00
9.976038e+00 7.309408e-03 5.264547e-05 1.000053e+00
  • I just found out this: Using exp(-1*sqrt($5*$5)) which although might be little expensive, still does the job fast enough for my KB sized files. Also, taking out the array format ${line[4]} was a lot of relief as I kept getting stuck with the syntax with that. Using straightforward printf is much more elegant. Thanks for you suggestions.. It would be of great help if you could add a few lines explaining how to add the titles to each column in my created data files.. – Aditya Putatunda Apr 18 '18 at 16:31
  • add NR==1{ print "N (k/(s/300.0000)) Abs(S)/E exp(Abs(S)/E)"} NR>1{.... – αғsнιη Apr 18 '18 at 17:03

If you want to do this with a python script:

#!/usr/bin/env python
# use: ingest.py infile1 infile2 ... infilen

import math,re,argparse

parser = argparse.ArgumentParser()
parser.add_argument('infiles', type=str, help='infile names', nargs='+')
args = parser.parse_args()

for infile in args.infiles:
    with open(infile) as in_file, open(infile+".out", "w+") as out_file:
        column_names = in_file.readline().split()
        out_file.write("N (k/(s/300.0000)) Abs(S)/E exp(Abs(S)/E)\n")
        for line in in_file.readlines():
            column_values = [float(x) for x in line.split()]
            value_dict = dict(zip(column_names,column_values))
            div = value_dict["k"]/(value_dict["s"]/300.0000)
            ABS = abs(value_dict["S"])/value_dict["E"]
            EXP = math.exp(abs(value_dict["S"])/value_dict["E"])
            out_file.write("{N} {DIV} {ABS} {EXP}\n".format(N=value_dict["N"],DIV=div,ABS=ABS, EXP=EXP))

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.