0

I've got a big .log file containing a lot of information and I would like to extract only few of this and put all in different outputs files.

Example of part of the .log file:

.....
New Water Solv 104: solv=  1.635

Saved WATERFLAP_REFINED2_SCORED_SOLV_H2O.pdb


Water:   1 AtId: 3021 ResId: 316    OH2n: -8.922    OH2s: -6.900    CRY: -0.640 ENTR: -0.321    SOLV:  0.000    DG:  6.636  CLASS: 1
Water:   2 AtId: 3013 ResId: 308    OH2n: -8.331    OH2s: -7.364    CRY: -0.885 ENTR: -0.321    SOLV:  0.000    DG:  6.453  CLASS: 1
Water:   3 AtId: 3009 ResId: 304    OH2n: -7.424    OH2s: -7.321    CRY:  5.000 ENTR: -0.036    SOLV:  0.577    DG:  5.450  CLASS: 1
Water:   4 AtId: 3064 ResId: 359    OH2n: -9.779    OH2s: -8.778    CRY: -1.187 ENTR: -0.804    SOLV:  0.000    DG:  3.279  CLASS: 1
Water: 103 AtId: 2996 ResId: 291    OH2n: -14.725   OH2s: -10.556   CRY: -1.060 ENTR: -0.607    SOLV:  0.962    DG: -0.849  CLASS: 5
Water: 104 AtId: 3004 ResId: 299    OH2n: -14.237   OH2s: -11.215   CRY: -1.197 ENTR: -0.500    SOLV:  1.635    DG: -1.185  CLASS: 5

Water Network Score Contributions:

Total       OH2n: -731.606  OH2s: -368.197  CRY: -30.908    ENTR: -94.714   DG:  28.882
Average     OH2n: -12.835   OH2s: -6.460    CRY: -0.542 ENTR: -1.662    DG:  0.507
Summary:     28.882 ( -10.345  39.228 )


Saved WATERFLAP_REFINED2_SCORED_OH2s_H2O.pdb

Saved WATERFLAP_REFINED2_SCORED_OH2n_H2O.pdb

Saved WATERFLAP_REFINED2_SCORED_DRY_H2O.pdb

Saved WATERFLAP_REFINED2_SCORED_CRY_H2O.pdb

Saved WATERFLAP_REFINED2_SCORED_ENTROPY_H2O.pdb

Saved WATERFLAP_REFINED2_SCORED_DG_WAT_H2O.pdb

Saved WATERFLAP_REFINED2_SCORED_CLASS_H2O.pdb

Saved WATERFLAP_REFINED2_SCORED_CLASS_COMPLEX.pdb

Saved WATERFLAP_REFINED2_SCORED.PDB

Saved WATERFLAP_REFINED2_SCORED_DG_WAT_H2O_ele.pdb

Saved WATERFLAP_REFINED2_SCORED_CLASS_H2O_ele.pdb

---------------------------
WaterFLAP summary of delta DG between apo and complex

Water: 1 AtId: 2994 ResId: 289  DG_APO: -6.921 DG_COMPLEX: -7.026 DDG: -0.105 CLASS: 3
Water: 2 AtId: 2995 ResId: 290  DG_APO: -1.789 DG_COMPLEX: -2.014 DDG: -0.225 CLASS: 3
Water: 3 AtId: 2996 ResId: 291  DG_APO: -0.841 DG_COMPLEX: -0.849 DDG: -0.008 CLASS: 3
Water: 121 AtId: 3138 ResId: 433    DG_APO:  0.000 DG_COMPLEX:  0.000 DDG:  0.000 CLASS: 3
Water: 122 AtId: 3143 ResId: 438    DG_APO:  0.000 DG_COMPLEX:  0.000 DDG:  0.000 CLASS: 3
Water_USED: 1 AtId: 2994 ResId: 289 OH2n: -15.983   OH2s: -15.953   CRY: -1.934 ENTR: -0.250    DG: -7.026  CLASS: 4
Water_USED: 2 AtId: 2995 ResId: 290 OH2n: -12.808   OH2s: -11.344   CRY: -0.291 ENTR: -0.411    DG: -2.014  CLASS: 4
Water_BOUNDARY: 3 AtId: 2996 ResId: 291 OH2n: -14.725   OH2s: -10.556   CRY: -1.060 ENTR: -0.607    DG: -0.849  CLASS: 5
Water_USED: 4 AtId: 2997 ResId: 292 OH2n: -14.971   OH2s: -14.678   CRY: -2.085 ENTR: -0.375    DG: -4.170  CLASS: 4
Water_BOUNDARY: 122 AtId: 3143 ResId: 438   OH2n:  5.000    OH2s: -0.110    CRY: -0.064 ENTR: -4.875    DG:  0.000  CLASS: 5

Saved WATERFLAP_Delta_DG_CLASS_H2O_ele.pdb

Saved WATERFLAP_Delta_DG_DG_WAT_H2O_ele.pdb

Saved WATERFLAP_Delta_DG_CLASS_H2O.pdb

Saved WATERFLAP_Delta_DG_DG_WAT_H2O.pdb


---------------------------

Apo:        DG:  35.441 DH: -3.791  -TDS:  39.232
Complex:    DG:  28.882 DH: -10.345 -TDS:  39.228

-------------
Net:        DG: -6.559  DH: -6.555  -TDS: -0.004


---------------------------
DG Displaced:   13.760
DDG Perturbed:  6.605
DG Disp-Pert:   7.155

---------------------------

WARNING: Setting ATOM parms from HETATM table
Atm:   CA  Q: 0.08
WARNING: Setting ATOM parms from HETATM table
Atm:   CA  Q: 0.08
....

The structure of the file is always the same but the number of rows, ID and numbers could change.

From this I would like to obtained 4 different outputs (maybe using some constant "string" always present as "Saved WATERFLAP_REFINED2_SCORED_SOLV_H2O.pdb", "Water Network Score Contributions", "WaterFLAP summary of delta DG between apo and complex", "Water Network Score Contributions:"???)

output1: (between "Saved WATERFLAP_REFINED2_SCORED_SOLV_H2O.pdb" and "Water Network Score Contributions")

Water:   1 AtId: 3021 ResId: 316    OH2n: -8.922    OH2s: -6.900    CRY: -0.640 ENTR: -0.321    SOLV:  0.000    DG:  6.636  CLASS: 1
Water:   2 AtId: 3013 ResId: 308    OH2n: -8.331    OH2s: -7.364    CRY: -0.885 ENTR: -0.321    SOLV:  0.000    DG:  6.453  CLASS: 1
Water:   3 AtId: 3009 ResId: 304    OH2n: -7.424    OH2s: -7.321    CRY:  5.000 ENTR: -0.036    SOLV:  0.577    DG:  5.450  CLASS: 1
Water:   4 AtId: 3064 ResId: 359    OH2n: -9.779    OH2s: -8.778    CRY: -1.187 ENTR: -0.804    SOLV:  0.000    DG:  3.279  CLASS: 1
Water: 103 AtId: 2996 ResId: 291    OH2n: -14.725   OH2s: -10.556   CRY: -1.060 ENTR: -0.607    SOLV:  0.962    DG: -0.849  CLASS: 5
Water: 104 AtId: 3004 ResId: 299    OH2n: -14.237   OH2s: -11.215   CRY: -1.197 ENTR: -0.500    SOLV:  1.635    DG: -1.185  CLASS: 5

Output2 (Between "WaterFLAP summary of delta DG between apo and complex" and first row starting with WATER_USED or WATER_BOUNDARY)

Water: 1 AtId: 2994 ResId: 289  DG_APO: -6.921 DG_COMPLEX: -7.026 DDG: -0.105 CLASS: 3
Water: 2 AtId: 2995 ResId: 290  DG_APO: -1.789 DG_COMPLEX: -2.014 DDG: -0.225 CLASS: 3
Water: 3 AtId: 2996 ResId: 291  DG_APO: -0.841 DG_COMPLEX: -0.849 DDG: -0.008 CLASS: 3
Water: 121 AtId: 3138 ResId: 433    DG_APO:  0.000 DG_COMPLEX:  0.000 DDG:  0.000 CLASS: 3
Water: 122 AtId: 3143 ResId: 438    DG_APO:  0.000 DG_COMPLEX:  0.000 DDG:  0.000 CLASS: 3

Output3 (Starting with row which contains WATER_USED or WATER_BOUNDARY and finished before Saved WATERFLAP_Delta_DG_CLASS_H2O_ele.pdb)

Water_USED: 1 AtId: 2994 ResId: 289 OH2n: -15.983   OH2s: -15.953   CRY: -1.934 ENTR: -0.250    DG: -7.026  CLASS: 4
Water_USED: 2 AtId: 2995 ResId: 290 OH2n: -12.808   OH2s: -11.344   CRY: -0.291 ENTR: -0.411    DG: -2.014  CLASS: 4
Water_BOUNDARY: 3 AtId: 2996 ResId: 291 OH2n: -14.725   OH2s: -10.556   CRY: -1.060 ENTR: -0.607    DG: -0.849  CLASS: 5
Water_USED: 4 AtId: 2997 ResId: 292 OH2n: -14.971   OH2s: -14.678   CRY: -2.085 ENTR: -0.375    DG: -4.170  CLASS: 4
Water_BOUNDARY: 122 AtId: 3143 ResId: 438   OH2n:  5.000    OH2s: -0.110    CRY: -0.064 ENTR: -4.875    DG:  0.000  CLASS: 5

Output4

Apo:        DG:  35.441 DH: -3.791  -TDS:  39.232
Complex:    DG:  28.882 DH: -10.345 -TDS:  39.228
Net:        DG: -6.559  DH: -6.555  -TDS: -0.004
DG Displaced:   13.760
DDG Perturbed:  6.605
DG Disp-Pert:   7.155

All Output should be a .txt file and the delimiter between columns (defined by "space" in the input files) should be "," or if it's simpler also "space" as in the input.

I have not idea how I can do that. Could someone help me with this hardest challenge please?

Improve this question
7
  • 1
    You say the delimiter between columns (defined by space in the input files) should be "," but that's not what you show in the expected output - please edit your question to show the exact output you expect given the input you provided.
    – Ed Morton
    Apr 29, 2022 at 19:47
  • This should be four separate questions (one per desired output). Or the OP should post his/her most pressing need, and then try to adapt posted Answers accordingly.
    – jubilatious1
    Apr 30, 2022 at 17:06
  • What be the conditions for Output4? Why only 3 lines of WATER_USED/BOUNDARY when there are 5 in your input?
    – RudiC
    May 1, 2022 at 8:35
  • Have you come across m4? you can set up multiple "diverts" to files and send data into the diverts. At the end you output the files, or even tie the diverts together into a report. There is quite a steep learning curve though!
    – user516667
    May 2, 2022 at 10:50
  • @RudiC I was wrong, I edited it!
    – Tommaso
    May 2, 2022 at 11:12

2 Answers 2

Reset to default
1 
$ cat tst.awk
/^Saved WATERFLAP_REFINED2_SCORED_SOLV_H2O.pdb/ { out = "output" 1; next }
/^Water Network Score Contributions/            { out = ""; next }
/^WaterFLAP summary of delta DG/                { out = "output" 2; next }
/^Water_(USED|BOUNDARY)/                        { out = ""; print > ("output" 3) }
/^(Apo|Complex|Net|DD?G)/                       { print > ("output" 4) }
out && NF { print > out }


$ awk -f tst.awk file


$ head out*
==> output1 <==
Water:   1 AtId: 3021 ResId: 316    OH2n: -8.922    OH2s: -6.900    CRY: -0.640 ENTR: -0.321    SOLV:  0.000    DG:  6.636  CLASS: 1
Water:   2 AtId: 3013 ResId: 308    OH2n: -8.331    OH2s: -7.364    CRY: -0.885 ENTR: -0.321    SOLV:  0.000    DG:  6.453  CLASS: 1
Water:   3 AtId: 3009 ResId: 304    OH2n: -7.424    OH2s: -7.321    CRY:  5.000 ENTR: -0.036    SOLV:  0.577    DG:  5.450  CLASS: 1
Water:   4 AtId: 3064 ResId: 359    OH2n: -9.779    OH2s: -8.778    CRY: -1.187 ENTR: -0.804    SOLV:  0.000    DG:  3.279  CLASS: 1
Water: 103 AtId: 2996 ResId: 291    OH2n: -14.725   OH2s: -10.556   CRY: -1.060 ENTR: -0.607    SOLV:  0.962    DG: -0.849  CLASS: 5
Water: 104 AtId: 3004 ResId: 299    OH2n: -14.237   OH2s: -11.215   CRY: -1.197 ENTR: -0.500    SOLV:  1.635    DG: -1.185  CLASS: 5

==> output2 <==
Water: 1 AtId: 2994 ResId: 289  DG_APO: -6.921 DG_COMPLEX: -7.026 DDG: -0.105 CLASS: 3
Water: 2 AtId: 2995 ResId: 290  DG_APO: -1.789 DG_COMPLEX: -2.014 DDG: -0.225 CLASS: 3
Water: 3 AtId: 2996 ResId: 291  DG_APO: -0.841 DG_COMPLEX: -0.849 DDG: -0.008 CLASS: 3
Water: 121 AtId: 3138 ResId: 433    DG_APO:  0.000 DG_COMPLEX:  0.000 DDG:  0.000 CLASS: 3
Water: 122 AtId: 3143 ResId: 438    DG_APO:  0.000 DG_COMPLEX:  0.000 DDG:  0.000 CLASS: 3

==> output3 <==
Water_USED: 1 AtId: 2994 ResId: 289 OH2n: -15.983   OH2s: -15.953   CRY: -1.934 ENTR: -0.250    DG: -7.026  CLASS: 4
Water_USED: 2 AtId: 2995 ResId: 290 OH2n: -12.808   OH2s: -11.344   CRY: -0.291 ENTR: -0.411    DG: -2.014  CLASS: 4
Water_BOUNDARY: 3 AtId: 2996 ResId: 291 OH2n: -14.725   OH2s: -10.556   CRY: -1.060 ENTR: -0.607    DG: -0.849  CLASS: 5
Water_USED: 4 AtId: 2997 ResId: 292 OH2n: -14.971   OH2s: -14.678   CRY: -2.085 ENTR: -0.375    DG: -4.170  CLASS: 4
Water_BOUNDARY: 122 AtId: 3143 ResId: 438   OH2n:  5.000    OH2s: -0.110    CRY: -0.064 ENTR: -4.875    DG:  0.000  CLASS: 5

==> output4 <==
Apo:        DG:  35.441 DH: -3.791  -TDS:  39.232
Complex:    DG:  28.882 DH: -10.345 -TDS:  39.228
Net:        DG: -6.559  DH: -6.555  -TDS: -0.004
DG Displaced:   13.760
DDG Perturbed:  6.605
DG Disp-Pert:   7.155
Improve this answer
2
  • Thanks it works, but I saw right now that the output 1 is wrong because the string "Saved WATERFLAP_REFINED2_SCORED_SOLV_H2O.pdb" and "Water Network Score Contributions" occur multiple time in the file, maybe 4 or more. I always need to extract last information, so all the strings between the last "Saved WATERFLAP_REFINED2_SCORED_SOLV_H2O.pdb" and "Water Network Score Contributions" reported in the input. Could you adapt the script for that please?
    – Tommaso
    May 2, 2022 at 15:00
  • If the sample input/output you provided doesn't match your real input/output such that it's not adequate to test a potential solution with then edit your question to fix your example. Actually, it'd probably be better if you ask a new, followup question for the case you describe since you already have multiple answers for the original example.
    – Ed Morton
    May 2, 2022 at 15:20
0

grep "string here" file.log

eg:

grep "Saved WATERFLAP_REFINED2_SCORED_SOLV_H2O.pdb" big.log

If you want line above or below the keyword add

-A NUM for after -B NUM for before -C NUM for before and after

If you want it to go to a file, use ">" to output the stdout to a txt file

eg:

grep -A 5 "Saved WATERFLAP_REFINED2_SCORED_SOLV_H2O.pdb" big.log > text.txt

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .