I've got a file (bigfile.txt), one of the columns looks like this

NW_017095471.1  Gnomon  mRNA    108321  109565  .   +   .   ID=rna34;Parent=gene27;Dbxref=GeneID:108565285,Genbank:XM_017925071.1;Name=XM_017925071.1;gbkey=mRNA;gene=LOC108565285;model_evidence=Supporting evidence includes similarity to: 7 Proteins%2C and 100%25 coverage of the annotated genomic feature by RNAseq alignments%2C including 30 samples with support for all annotated introns;product=transmembrane protein 126A;transcript_id=XM_017925071.1

I have a separate list:


I want to take each line and grep the ID field and then return the 'LOC#' that follows "Name=".

`grep $gene";" bigfile.txt | sed -e 's/Name=

To return


How do I go about extracting just this portion?

  • Tried this? awk -F'[; =]' '/ID=gene27/{print $6}' infile – Valentin Bajrami Jul 11 '19 at 6:48
  • @ValentinBajrami but there are multiple genes stored in a different file. – Prvt_Yadav Jul 11 '19 at 6:49
  • @Prvt_Yadv don't know where you are reading that but maybe we just should wait for the OP to clarify his/her goal` – Valentin Bajrami Jul 11 '19 at 6:51
  • @ValentinBajrami well I am not OP, but as he wrote in question, there is a separate list, so I am assuming there are multiple vallues. – Prvt_Yadav Jul 11 '19 at 6:52
  • yes, exactly take the multiple values (e.g. each line in list) and search the bigfile to extract LOC. In my example I have 3 lines and I just want 2 LOC id's ... does this make sense? – R-MASHup Jul 11 '19 at 6:54

Try this,

    grep "ID=$pattern;" file | grep -o 'Name=[^;]*' | sed 's/Name=//g';
  • grep -o 'Name=[^;]*' with grep the pattern starts with "Name=" and immediate ends ";"
  • sed 's/Name=//g'; just remove the pattern which is not required as per your requirement.
| improve this answer | |

Assuming this is the 9th tab-delimited field of a GFF file (the "attributes" field), you can extract the value of the gene attribute corresponding to a particular ID attribute (read from a separate file) like this with awk:

BEGIN { FS = "\t" }

FNR == NR {
    # Read IDs into a hash as keys.
    ids[$1] = 1

$3 == "gene" {
    # Split the attribute field into separate key-value pairs.
    n = split($9, keyvalues, ";")

    id = ""    # Not found a gene ID yet
    gene = ""  # No gene name to print

    # Loop over the key-value pairs, split them on the "="
    # and extract the gene name and gene ID.
    for (i = 1; i <= n; ++i) {
        split(keyvalues[i], attr, "=")
        if (attr[1] == "ID") {
            if (attr[2] in ids)
                id = attr[2]
                next  # This line is not of interest
        else if (attr[1] == "gene")
            gene = attr[2]

    if (id != "" && gene != "")
        print gene

Running this on a GFF file called file.gff containing the given data in column 9, and with the gene ID list in id.list:

$ awk -f script.awk id.list file.gff

The list of gene IDs are read from the first file in the FNR == NR block in the awk code, while the last block is processing the attribute field of gene feature lines (only) in the second (and all later) files given on the command line.

The awk code assumes that the ID and gene attributes of the GFF file only contains a single value (not a comma-delimited list of values) and that the values are not quoted.

To get the output as a list of gene names and gene IDs (two columns), change the print gene statement to print id, gene.

| improve this answer | |

This needs refactoring but should do what you want:

while IFS=; read -r line; do grep -Fw "$line" biffile.txt; done < other_file | awk -F';' '{split($3,a,"=");print a[2]}'
| improve this answer | |
  • You can use grep -f pat.list to grep for patterns read from a file, but your grep may get stuff it does not want, like a like containing gene1000 if the pattern is gene10 (you have the same issue in the current code). – Kusalananda Jul 11 '19 at 7:36
  • @Kusalananda that's a good point. I guess this can be solved by passing -w to grep as in ..grep -Fw – Valentin Bajrami Jul 11 '19 at 7:46
  • Yes, possibly, if we assume that the strings in the input list are words and don't start or end with a non-word character. – Kusalananda Jul 11 '19 at 7:57

I would use a slightly different approach. First, extract only the ID and Name fields:

$ sed -nE 's/.*ID=([^;]*).*Name=([^;]*).*/\1 \2/p' file1
gene27 LOC108565285
gene28 LOC108569527
gene78 LOC108562956

Then, filter that using the list of target IDs:

$ cat file2
$ sed -nE 's/.*ID=([^;]*).*Name=([^;]*).*/\1 \2/p' file1 | grep -wf file2 
gene27 LOC108565285
gene28 LOC108569527

Or, if you only want the LOC.... value, and assuming you have GNU grep:

$ grep -wf file2 file1 | grep -oP 'Name=\K[^;]+'
| improve this answer | |

Safe. Simple. Short.

The name bigfile.txt suggests that we don't want to grep multiple times through it, but do it in one pass. So I prefer an attempt like @Kusalananda: Collect all gene IDs to extract and than scan once through the bigfile. For such a task I prefer sed, as you don't need do deal with programming like variables, arrays and loops, just let the tool do the work:

sed '/^gene[0-9]*$/{s/$/;/;H;d;};G;/ID=\(gene[0-9]*;\).*\n\1/!d;s/.*gene=\(LOC[0-9]*\);.*/\1/' genelist.txt bigfile.txt

And here is the explanation:

  • /^gene[0-9]*$/ is a pattern to select lines of the list: just gene with a number
  • {s/$/;/;H;d;} is executed only for the lines above: s/$/;/ appends a ;, H appends this modified line to the hold space and d stops further processing and deletes the line
  • At this point in the script we know we are in the bigfile.txt and the hold space is filled with all geneIDs. Now G appends this list to the pattern space
  • The pattern /ID=\(gene[0-9]*;\).*\n\1/ selects lines where the ID gene[0-9]* is repeated (the \1 refers to the string inside the first pair of \(\)) after a newline, thus an ID that is present in the list
  • !d inverts the selection and deletes, to all lines without a match get deleted
  • Now for all line with a match, extract the gene field: s/.*gene=\(LOC[0-9]*\);.*/\1/
| improve this answer | |

Done by below simple script


for i in `cat file2`; do awk -v i="$i" -F ";" '$1=="ID="i{print $5}' file1| awk -F  "=" '{print $NF}'; done


| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.