Expertos: I want to extract fields based on a string (partial and exact). In some rows these fields are moved one or two columns, so I can't specify the column (which is the extent of my knowledge). I also need to extract the 1st, 2nd, 4th and 5th fields, but those don't vary row to row, so I can do that myself.

Specifically, I want to extract the field having the partial string "DP4=" and the field with a matching string like "MODERATE" plus the field to the left and right:


NW_006532014.1  603822  .       T       C       222.0   .       DP=103  ADF=30,22       ADR=13,16       AD=43,38        VDB=0.0570121   SGB=-0.693143   RPB=0.810487    MQB=0.570226      MQSB=0.033126   BQB=0.964281    MQ0F=0  ICB=1   HOB=0.5 AC=1    AN=2    DP4=30,13,22,16 MQ=35   ANN=C   missense_variant        MODERATE        ABCB6   ABCB6     transcript      XM_007419806.2
NW_006532015.1  1015800 .       AAA     AAACAA  228.0   .       INDEL   IDV=106 IMF=0.905983    DP=117  ADF=6,50        ADR=19,42       AD=25,92        VDB=0.22041     SGB=-0.693147     MQSB=0.182586   MQ0F=0  AC=2    AN=2    DP4=6,19,50,42  MQ=36   ANN=AAACAA      disruptive_inframe_insertion    MODERATE        CEP131  CEP131  transcriptXM_025166060.1  protein_coding  11/27
NW_006532017.1  910856  .       C       G       78.0    .       DP=118  ADF=50,14       ADR=25,2        AD=75,16        VDB=0.954018    SGB=-0.689466   RPB=0.939107    MQB=0.0031569     MQSB=0.280595   BQB=0.0859367   MQ0F=0  ICB=1   HOB=0.5 AC=1    AN=2    DP4=50,25,14,2  MQ=44   ANN=G   missense_variant        MODERATE        HPS1    HPS1      transcript      XM_025169525.1
NW_006532033.1  475415  .       T       C       222.0   .       DP=123  ADF=18,36       ADR=18,15       AD=36,51        VDB=0.984451    SGB=-0.693147   RPB=0.428811    MQB=1.68713e-05   MQSB=0.565818   BQB=0.837943    MQ0F=0  ICB=1   HOB=0.5 AC=1    AN=2    DP4=18,18,36,15 MQ=41   ANN=C   missense_variant        MODERATE        PTCH2   PTCH2     transcript      XM_025164053.1
NW_006532040.1  586236  .       G       C       29.9689 .       DP=106  ADF=40,9        ADR=25,1        AD=65,10        VDB=0.731987    SGB=-0.670168   RPB=0.945403    MQB=0.00509228    MQSB=0.210814   BQB=0.365756    MQ0F=0  ICB=1   HOB=0.5 AC=1    AN=2    DP4=40,25,9,1   MQ=46   ANN=C   missense_variant        MODERATE        OMG     OMG       transcript      XM_007420376.3
NW_006532040.1  674528  .       T       C       221.0   .       DP=128  ADF=39,15       ADR=19,9        AD=58,24        VDB=0.253732    SGB=-0.692831   RPB=0.952839    MQB=1.09944e-10   MQSB=0.755937   BQB=0.749586    MQ0F=0  ICB=1   HOB=0.5 AC=1    AN=2    DP4=39,19,15,9  MQ=48   ANN=C   missense_variant        MODERATE        NF1     NF1       transcript      XM_007420379.3

desired output (tab-delim):

DP4=30,13,22,16 missense_variant        MODERATE        ABCB6
DP4=6,19,50,42  disruptive_inframe_insertion    MODERATE        CEP131
DP4=50,25,14,2  missense_variant        MODERATE        HPS1
DP4=18,18,36,15 missense_variant        MODERATE        PTCH2
DP4=40,25,9,1   missense_variant        MODERATE        OMG
DP4=39,19,15,9  missense_variant        MODERATE        NF1



2 Answers 2


Using GNU awk for the 3rd arg to match(), \< word boundary, and \s/\S shorthand:

$ awk -v OFS='\t' 'match($0,/(\<DP4=\S+).*\s(\S+\tMODERATE\t\S+)/,a){print a[1], a[2]}' file
DP4=30,13,22,16 missense_variant        MODERATE        ABCB6
  • Second parenthesis: .....\s+MODERATE\s+.....i.e., \s+ instead of \t, perhaps? Oct 8, 2021 at 7:03
  • @CarlosPascual not sure what you mean by "parenthesis" there. I considered \s or \s+ instead of \t but the OP said their input is tab-separated so it shouldn;t matter and at one point I was thinking \t was better for accommodating blanks within fields but it really doesn't so I just left it with \ts anyway.
    – Ed Morton
    Oct 8, 2021 at 13:21
  • ok. understood. I refer only to the second parenthesis of the regexp. Testing your script with \s+ instead of \t (\S+\s+MODERATE\s+\S+) I get the desired result; but not with (\S+\tMODERATE\t\S+). Oct 8, 2021 at 15:05
  • 1
    @CarlosPascual then your field separators arent tabs as the OP specified. You probably copy/pasted the posted example and forgot to change the chains of blanks to tabs.
    – Ed Morton
    Oct 8, 2021 at 15:16
  • 1
    yes, you are rigth, `I forgot to change the chains of blanks to tabs. Oct 8, 2021 at 15:21
awk 'BEGIN{ OFS="\t" } {
    nrf=split($0, tmp); s1=s2=0;
    for(i=1; i<=nrf; i++){
        printf "%s", (tmp[i] ~/DP4=/     &&++s1? (s2?OFS:"") tmp[i]:
                     (tmp[i]=="MODERATE" &&++s2? (s1?OFS:"") tmp[i-1] OFS tmp[i] OFS tmp[i+1]:"") );
    }; print "";
}' infile

we used the split() function to split the current processing line each time to a temporary array called tmp on default FS (whitespaces, i. e. on Tabs/Spaces); the nrf is just a temporary variable I used and that holds the number of fields that the split() function did splitting.

then we used a for-look on those fields and checking if the current field tmp[i] which read meet the conditions as you are expecting, if it was we then printing else we are checking the next condition if that was seen then print one previous field to it tmp[i-1] then the current field itself tmp[i] then the right hand next field to it tmp[i+1], otherwise we print empty string "".

temporary variables s1 and s2 are used to control the fields delimiter between first and second print conditions action, so if one was found before the next fields should print OFS before in advance.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .