0

I have a tab-separated multi-column file in which column 19 is like this:

gaA
gGg
Att
gtC
gGa
gcC
ccG
cTc
.
.
.
and so on

I want to grep uppercase characters only, so I used:

cut -f19 1.table | grep -e '[[:upper:]]' -o

The output is:

A
G
A
C
G
C
G
T
.
.
.
and so on

But I do not want to use cut before grep. I have two questions now:

  1. Is there any way to grep from column 19 rather than using cut? or is there any option or parameter in grep to specify a column?
  2. I want to put the grep result output as a new column in 1.table file? or how to pipe the grep result output as a new column (as column 20) in 1.table file?

Here are the input lines from 1.table (1.table has header as well):

#CHROM  POS ID  REF ALT QUAL    AC  AN  AF  DP  ExonicFunc.refGene  Func.refGene    AAChange.refGene    Gene.refGene    GeneDetail.refGene  GENEINFO    EFF refcodon            ExAC_AFR    ExAC_ALL    ExAC_AMR    ExAC_EAS    ExAC_FIN    ExAC_NFE    ExAC_OTH    ExAC_SAS    gnomAD_exome_AFR    gnomAD_exome_ALL    gnomAD_exome_AMR    gnomAD_exome_ASJ    gnomAD_exome_EAS    gnomAD_exome_FIN    gnomAD_exome_NFE    gnomAD_exome_OTH    gnomAD_exome_SAS    gnomAD_genome_AFR   gnomAD_genome_ALL   gnomAD_genome_AMR   gnomAD_genome_ASJ   gnomAD_genome_EAS   gnomAD_genome_FIN   gnomAD_genome_NFE   gnomAD_genome_OTH   1000g2015aug_all    esp6500siv2_all CADD_phred  CADD_raw    CADD_raw_rankscore  CAF DANN_rankscore  DANN_score  Eigen   Eigen-PC-raw    Eigen-raw   Eigen_coding_or_noncoding   FATHMM_coding   FATHMM_converted_rankscore  FATHMM_noncoding    FATHMM_pred FATHMM_score    FS  GTEx_V6_gene    GTEx_V6_tissue  GWAVA_region_scoreGWAVA_tss_score   GWAVA_unmatched_score   GenoCanyon_score    GenoCanyon_score_rankscore  Interpro_domain LRT_converted_rankscore LRT_pred    LRT_score   MetaLR_pred MetaLR_rankscore    MetaLR_score    MetaSVM_pred    MetaSVM_rankscore   MetaSVM_score   MutationAssessor_pred   MutationAssessor_score  MutationAssessor_score_rankscore    MutationTaster_converted_rankscore  MutationTaster_pred MutationTaster_score    PROVEAN_converted_rankscore PROVEAN_pred    PROVEAN_score   Polyphen2_HDIV_pred Polyphen2_HDIV_rankscore    Polyphen2_HDIV_score    Polyphen2_HVAR_pred Polyphen2_HVAR_rankscore    Polyphen2_HVAR_score    QD  SIFT_converted_rankscore    SIFT_pred   SIFT_score  SiPhy_29way_logOdds SiPhy_29way_logOdds_rankscore   VC  VEST3_rankscore VEST3_score WGT avsnp147=rs28410799 integrated_confidence_value integrated_fitCons_score    integrated_fitCons_score_rankscorephastCons100way_vertebrate    phastCons100way_vertebrate_rankscore    phastCons20way_mammalian    phastCons20way_mammalian_rankscorephyloP100way_vertebrate   phyloP100way_vertebrate_rankscore   phyloP20way_mammalian   phyloP20way_mammalian_rankscore CLINSIG CLNACC  CLNDBN  CLNDSDB CLNDSDBID   GT  AD  DP  GQ  PL  GT  AD  DP  GQ  PL  GT  AD  DP  GQPL    
chr1    13115765    rs141111983 C   T   2280.92 3   6   0.5 153 synonymous_SNV  exonic  HNRNPCL2:NM_001136561:exon2:c.G636A:p.E212E HNRNPCL2    0   HNRNPCL2:440563 SYNONYMOUS_CODING(LOW|SILENT|gaG/gaA|E212|293|HNRNPCL2|protein_coding|CODING|ENST00000621994|2|T),NEXT_PROT[coiled-coil_region](LOW||||293|HNRNPCL2|protein_coding|CODING||2|T),INTRON(MODIFIER||||478|WI2-3308P17.2|protein_coding|CODING|ENST00000622351|1|T),INTRON(MODIFIER||||478|PRAMEF26|protein_coding|CODING|ENST00000621259|4|T)  gaG gaA E212    0.4772  0.4933  0.4993  0.497   0.5 0.4918  0.4967  0.4996  0.4846  0.4959  0.4998  0.4969  0.499   0.4999  0.4939  0.4969  0.4998  0.4939  0.4125  0.4867  0.1888  0.4981  0.4997  0.3321  0.4604  0   0   0   0   0   0   0   0-0.3847    -0.3847-PC-raw  -0.3847-raw 0   0.02308 0   0.87915 0   0   18.131  0   0   0.43    0.23    182 000 0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   000 0   0   0   0   0   14.91   0   0   0   0   0   SNV 0   0   1   rs141111983=rs28410799  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0B00H7EW=0/1    38,18   56  99  722,0,1577  B00H7EX=0/1 31,29   60  99  1166,0,1211 B00H7EY=0/1 26,11   3799    423,0,1098  
chr1    13115766    rs150951326 T   C   2325.92 3   6   0.5 155 nonsynonymous_SNV   exonic  HNRNPCL2:NM_001136561:exon2:c.A635G:p.E212G HNRNPCL2    0   HNRNPCL2:440563 NON_SYNONYMOUS_CODING(MODERATE|MISSENSE|gAg/gGg|E212G|293|HNRNPCL2|protein_coding|CODING|ENST00000621994|2|C),NEXT_PROT[coiled-coil_region](LOW||||293|HNRNPCL2|protein_coding|CODING||2|C),INTRON(MODIFIER||||478|WI2-3308P17.2|protein_coding|CODING|ENST00000622351|1|C),INTRON(MODIFIER||||478|PRAMEF26|protein_coding|CODING|ENST00000621259|4|C)  gAg gGg E212G   0.4775  0.4934  0.4993  0.4972  0.5 0.4919  0.4967  0.4996  0.4851  0.496   0.4998  0.4969  0.4991  0.4999  0.494   0.4969  0.4998  0.494   0.4127  0.4867  0.1875  0.4981  0.4997  0.3323  0.4603  0   0   0.286   -0.453  0.058   0   0.019   0.324   -0.4897 -0.4897-PC-raw  -0.4897-raw n   0.01015 0   0.80402 0   0   22.504  0   00.43   0.23    182 0   0.029   0   0   0   0   0   0   0   0   0   0   0   0   000 0   0   0   0   B   0.026   0   B   0.013   0   15.01   0   0   0   0   0   SNV 0.089   0.091   1   rs150951326=rs28410799  0   0.075   0.013   0.947   0.327   0.005   0.09    -0.854  0.044   -0.972  0.023   0   0   0   0   0   B00H7EW=0/1 38,19   57  99  764,0,1574  B00H7EX=0/1 31,30   61  991166,0,1211   B00H7EY=0/1 26,11   37  99  426,0,1056  
chr1    13392320    rs767291041 C   A   96.12   1   4   0.25    10  nonsynonymous_SNV   exonic  PRAMEF16:NM_001045480:exon3:c.C1243A:p.P415T,PRAMEF17:NM_001099851:exon3:c.C1243A:p.P415T   PRAMEF16,PRAMEF17   0   PRAMEF17:391004 NON_SYNONYMOUS_CODING(MODERATE|MISSENSE|Cct/Act|P415T|474|PRAMEF17|protein_coding|CODING|ENST00000376098|3|A)   Cct Act P415T   0   0.001   0   0   0   0.002   0   0   0   0.0006  0.0016  0   0   0   0.0009  0.0008  0   0   0.0002  0   0   0   0   0.0007  0   0   0   0   0   0   0   0   0   -0.1177 -0.1177-PC-raw  -0.1177-raw 0   0.02548 0   0.24739 0   0   0   0   0   0   0   0   0   0   000 0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   000 0   0   0   10.68   0   0   0   0   0   SNV 0   0   1   rs782058522=rs28410799  000 0   0   0   0   0   0   0   0   0   0   0   0   0   B00H7EW=0/1 4,5 999 123,0,100   B00H7EX=0/0 1,0 1   3   0,3,30  B00H7EY=./. 0,0 0   0   0,0,0   
chr1    13392320    rs767291041 C   A   70.13   1   6   0.167   37  nonsynonymous_SNV   exonic  PRAMEF17:NM_001099851:exon3:c.C1243A:p.P415T    PRAMEF17    0   PRAMEF17:391004 NON_SYNONYMOUS_CODING(MODERATE|MISSENSE|Cct/Act|P415T|474|PRAMEF17|protein_coding|CODING|ENST00000376098|3|A)   Cct Act P415T   0.0048  0.0006  0   0   0   0   0   00.0002 0.0005  0.0009  0   0   0   0.0007  0.0006  0   0   0.0002  0   0   0   0   0.0004  0   0022.7  3.18    0.442   0   0.453   0.988   -0.1208 -0.1208-PC-raw  -0.1208-raw c   0.0207  0.12    0.11274 T   2.72    000 0   0   0   0   0.061   0   0.843   D   0   T   0.356   0.094   T   0.045   -1.097  H   3.83    0.957   0.09    N   1   0.954   D   -7.63   D   0.899   1   D   0.875   0.998   11.69   0.784   D   0.001   5.599   0.165   SNV 0.353   0.293   1   rs767291041=rs28410799  0   0.487   0.133   0.019   0.194   0.031   0.148   1.655   0.367   0.621   0.289   0   0   0   0   0   B00H7EW=0/1 3,3 6   94  101,0,94    B00H7EX=0/0 13,0    13  39  0,39,442    B00H7EY=0/0 18,0    18  48  0,48,720
  • 1
    Any particular reason not to use cut? This is precisely what cut is for, after all. And pelase edit your question and give us an example of your input file and the output you want to see from it. How should the column be added? After the 19th? At the end of the line? At the beginning? – terdon Apr 24 '17 at 10:59
  • I have made necessary changes to the question. – Waqasuddin Khan Apr 24 '17 at 12:18
  • 1
    Is that a tab separated file? There are no tabs in your example. What should happen to the original column 20? Should it become 21? I mean, should all columns be shifted to the right? Please add your desired output as requested. – terdon Apr 24 '17 at 12:32
  • What should the outcome be if there's no uppercase character or more than 1 in the 19th field? – Stéphane Chazelas Apr 24 '17 at 14:37
4

is there any option or parameter in grep to specify a column?

grep doesn't have field separator option.
Use the following awk approach instead:

awk -F'\t' -v OFS='\t' '{match($19,/[A-Z]+/); $20=substr($19,RSTART,RLENGTH) FS $20}1' 1.table

match($19,/[A-Z]+/) - captures uppercase letters within 19th field

$20=substr($19,RSTART,RLENGTH) FS $20 - extracts matched uppercase letters from 19th field and inserts it as 20th field value

  • I tried with you command, but it gives an error: awk: line 1: syntax error at or near , is there any other information which I want to mention. – Waqasuddin Khan Apr 24 '17 at 10:55
  • the error is verbose, check your command for syntax typos. Also, it would be better if you add few lines from your 1.table into the question – RomanPerekhrest Apr 24 '17 at 10:58
  • @Roman You have to define also OFS="\t" , since your solution as presented will convert the tab delimited file to a space delimited file (default OFS is space). – George Vasiliou Apr 24 '17 at 11:12
  • @GeorgeVasiliou, sure, just missed for a moment. Still expect to see the input example – RomanPerekhrest Apr 24 '17 at 11:14
  • 1
    @WaqasuddinKhan If you are happy with this answer you might accept it as a solution by pressing the "tick mark". – George Vasiliou Apr 24 '17 at 14:16
1

To answer your literal question about how to do it with grep alone. Even though grep has not been designed for that, with GNU grep and built with PCRE support, you could do:

grep -Po '(?:^(?:[^\t]*\t){18}|\G)[^\t]*?\K[[:upper:]]'

That is search for either 18 sequences of <not-TABs><tab> at the beginning of the line or the end of the previous match (\G) followed by the smallest number possible of non-tab characters (so we're still in the 19th field) followed by an upper case caracter, but with \K we reset the start of the matched portion just before that uppercase character.

So on an input like:

X<tab>X<tab>....<tab>AbC<tab>X<tab>...

It would report:

A
C

Like with your cut | grep approach.

If you're only interested in the first uppercase character in the 19th field, you can simplify it to:

grep -Po '^(?:[^\t]*\t){18}[^\t]*?\K[[:upper:]]'

To insert it as the 20th column, you could do:

paste <(cut -f1-19 < file) <(grep ...above < file) <(cut -f20- < file) > newfile

Or to insert it as the last column:

grep... < file | paste file - > newfile
0

With sed you can do

sed '/^#/!s/\([^ ]* *\)\{18\}[a-z]*\([A-Z]\).*/&  \2/'

This is, for all lines not starting with # (the /^#/! selector), after 18 combinations of non-spaces and spaces, mark the uppercase letter with \(\) to later refer to it, "replace" the whole line by itself and append spaces with the found uppercase letter.

If you prefer extended regular expressions, you can as well use

sed -E '/^#/!s/([^ ]* *){18}[a-z]*([A-Z]).*/&  \2/'

If columns are separated by a tab instead of spaces, you go

sed -E '/^#/!s/([^\t]*\t){18}[a-z]*([A-Z]).*/&\t\2/'

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.