4

The original output file contained this block of text among much more information:

 Projecting out rotations and translations

 Force Constants (Second Derivatives of the Energy) in [a.u.]
                             GX1         GY1         GZ1         GX2         GY2     
           GX1           0.6941232
           GY1           0.0187624   0.0156533
           GZ1          -0.1175495  -0.0980708   0.6144300
           GX2          -0.6074291  -0.0036667   0.0229726   0.6228918
           GY2           0.0069881  -0.0013581   0.0085087   0.0023190   0.0014047
           GZ2          -0.0437815   0.0085087  -0.0533084  -0.0145287  -0.0088007
           GX3          -0.0866941  -0.0150957   0.0945769  -0.0154627  -0.0093070
           GY3          -0.0257505  -0.0142952   0.0895621   0.0013477  -0.0000466
           GZ3           0.1613309   0.0895621  -0.5611216  -0.0084438   0.0002920
                             GZ2         GX3         GY3         GZ3     
           GZ2           0.0551377
           GX3           0.0583102   0.1021568
           GY3           0.0002920   0.0244027   0.0143418
           GZ3          -0.0018293  -0.1528871  -0.0898540   0.5629509

So far I have managed to isolate the data I need along with the relevant headings, and print this to a log file using [grep] and [awk] (below):

#!/bin/bash

rm Hessian.log

for i  in *.out
do
grep -H -A16 "Force Constants (Second Derivatives of the Energy)" $i | tail -n +1 | awk ' NR == 2 {printf "     "" %10s %10s %10s %10s %10s \n", $2,$3,$4,$5,$6} NR == 3, NR == 11 {printf "%5s %10s %10s %10s %10s %10s\n", $2,$3,$4,$5,$6,$7} ' >> Hessian.log
echo "" >> Hessian.log
done

Which produces:

          GX1         GY1         GZ1         GX2         GY2     
GX1    0.6941232
GY1    0.0187624   0.0156533
GZ1   -0.1175495  -0.0980708   0.6144300
GX2   -0.6074291  -0.0036667   0.0229726   0.6228918
GY2    0.0069881  -0.0013581   0.0085087   0.0023190   0.0014047
GZ2   -0.0437815   0.0085087  -0.0533084  -0.0145287  -0.0088007
GX3   -0.0866941  -0.0150957   0.0945769  -0.0154627  -0.0093070
GY3   -0.0257505  -0.0142952   0.0895621   0.0013477  -0.0000466
GZ3    0.1613309   0.0895621  -0.5611216  -0.0084438   0.0002920
          GZ2         GX3         GY3         GZ3     
GZ2    0.0551377
GX3    0.0583102   0.1021568
GY3    0.0002920   0.0244027   0.0143418
GZ3   -0.0018293  -0.1528871  -0.0898540   0.5629509

However, I am trying to move the last four lines so that they sit in columns next to the data above, with their respective headings (GZ2, GX3, GY3, GZ3) in the same row as the other headings. To put it simply, the resulting output should be a 9*9 matrix of data with labels for each column and row (as shown below).

          GX1         GY1         GZ1         GX2         GY2         GZ2         GX3         GY3         GZ3
GX1    0.6941232
GY1    0.0187624   0.0156533
GZ1   -0.1175495  -0.0980708   0.6144300
GX2   -0.6074291  -0.0036667   0.0229726   0.6228918
GY2    0.0069881  -0.0013581   0.0085087   0.0023190   0.001404
GZ2   -0.0437815   0.0085087  -0.0533084  -0.0145287  -0.0088007   0.0551377
GX3   -0.0866941  -0.0150957   0.0945769  -0.0154627  -0.0093070   0.0583102   0.1021568
GY3   -0.0257505  -0.0142952   0.0895621   0.0013477  -0.0000466   0.0002920   0.0244027   0.0143418
GZ3    0.1613309   0.0895621  -0.5611216  -0.0084438   0.0002920  -0.0018293  -0.1528871  -0.0898540   0.5629509
  • The word "Data" instead of actual data makes it somewhat unclear what you want here. – Wildcard Dec 7 '16 at 23:36
  • my bad, I was just limited by the space I was typing in, 'data' is supposed to represent all the numbers in the final four lines of the original text. – Stephen Mason Dec 7 '16 at 23:37
  • Code blocks scroll sideways. I would recommend editing your question to include that data. – Wildcard Dec 7 '16 at 23:44
  • 1
    "The output was originally not this clear, I have got to this point using awk:" Could this be an X-Y problem? We can certainly help you solve this portion of your problem, but have you considered sharing the original data (and your complete formatting requirements) in case we can help you get a better overall result? Ultimately it's up to you, but it might be worth it. Right now there's only one answer, and it's mine, so an edit wouldn't be out of the question at this point. – type_outcast Dec 8 '16 at 1:44
  • 1
    OK, thanks for posting the original input. It looks very much like what you had before, only with some extraneous text and leading whitespace. It would be easy to adapt my answer to accommodate that so the entire solution is in one script. I can do that if you like, or, since you're learning, you may want to tackle that as a learning exercise, since it should be fairly straightforward, and we're here to help if you get stuck. :-) – type_outcast Dec 8 '16 at 2:24
1

A simple Perl script will do the trick nicely (Perl is already installed on dang near everything):

#!/usr/bin/env perl

my @rows; # Preserve order of appearance
my %rows;

my $heading;

for (<>) {
    chomp;
    if (s/^\s+/   /) {
        $heading .= $_;
    } elsif (/^(\w+) (.*)$/) {
        push @rows, $1 if not exists $rows{$1};
        $rows{$1} .= $2;
    } else {
        die "Invalid line format at line $.";
    }
}
my $fmt = "%-5s %s\n"; # Adjust width to suit taste
printf $fmt, '', $heading;
printf $fmt, $_, $rows{$_} for @rows;

Simply invoke this program with your data something like so:

$ my_column.pl < your_data.txt

(Assuming you saved the above script as my_column.pl and made it executable with chmod 755 my_column.pl of course!)

The above should get the job done, but if you need precise column alignment or more advanced formatting in general, you can split the columns and force particular column widths with printf, or one of the many tabular formatting modules available for Perl.

  • Thank you for such a comprehensive answer! However, the output I have provided in the question was produced using awk in a bash script, (editing my question now!) could you possibly break your script down so that I can attempt to recreate it using bash? – Stephen Mason Dec 8 '16 at 1:28
  • Sure! I can give it a try. You've mentioning awk, bash and you had a shell-scripting tag before you removed it in the last edit, so it would help if you could explain which tools you need to use, and maybe even a quick blurb on why you can use awk but not, say, perl. Even if it's just your personal preference, that'd be fine of course. – type_outcast Dec 8 '16 at 1:41
  • 1
    No problem (I hadn't even noticed somebody had edited my tags!), and yeah basically I am relatively new to Linux and had to look up what perl was after reading your answer! Only just getting my head around the basics of bash scripting, I found awk a very useful tool for tasks like this where I need to reorganise text based on columns and rows etc, however my lack of experience with regular expressions seems to be holding me up. – Stephen Mason Dec 8 '16 at 1:47
  • 1
    Fair enough. Welcome to Linux! With this sort of question, there are many ways to arrive at the correct answer. I'm certainly not going to sit here and try to convince you that one is better than another, but for this problem, today, for me, Perl seemed like a good fit. If you want to learn regular expressions, Perl's implementation is a really good one to learn, and it's very well documented. – type_outcast Dec 8 '16 at 2:06
0

Managed to solve my own problem by simply assigning the specific line and column as a variable, and concatenating them using echo, simple when you know the answer!

#!/bin/bash

cd FREQ/HF
rm Hessian.log


for i  in *.out
do
grep -H -A16 "Force Constants (Second Derivatives of the Energy)" $i | tail -n +1 >> Hessian.tmp

x=`awk ' NR == 2 {printf "     "" %10s %10s %10s %10s %10s \n", $2,$3,$4,$5,$6}' Hessian.tmp`
y=`awk ' NR == 12 {printf "%10s %10s %10s %10s \n", $2,$3,$4,$5}' Hessian.tmp`
a=`awk ' NR == 8 { printf "%5s %10s %10s %10s %10s %10s\n", $2,$3,$4,$5,$6,$7} ' Hessian.tmp`
b=`awk ' NR == 9 { printf "%5s %10s %10s %10s %10s %10s\n", $2,$3,$4,$5,$6,$7} ' Hessian.tmp`
c=`awk ' NR == 10 { printf "%5s %10s %10s %10s %10s %10s\n", $2,$3,$4,$5,$6,$7} ' Hessian.tmp`
d=`awk ' NR == 11 { printf "%5s %10s %10s %10s %10s %10s\n", $2, $3,$4,$5,$6,$7} ' Hessian.tmp`
e=`awk ' NR == 13 { printf "%10s", $3} ' Hessian.tmp`
f=`awk ' NR == 14 { printf "%10s %10s", $3, $4} ' Hessian.tmp`
g=`awk ' NR == 15 { printf "%10s %10s %10s", $3, $4,$5} ' Hessian.tmp`
h=`awk ' NR == 16 { printf "%10s %10s %10s %10s", $3, $4, $5,$6} ' Hessian.tmp`

echo "$x $y" >> Hessian.log
awk ' 
NR == 3, NR == 7 {printf "%5s %10s %10s %10s %10s %10s\n", $2,$3,$4,$5,$6,$7} ' Hessian.tmp >> Hessian.log
echo "$a $e" >> Hessian.log
echo "$b $f" >> Hessian.log
echo "$c $g" >> Hessian.log
echo "$d $h" >> Hessian.log
rm Hessian.tmp
echo "" >> Hessian.log
done

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.