Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a file (>80,000 lines) that looks likes this:

chr1    GTF2GFF chromosome  1   249213345   .   .   .   ID=chr1;Name=chr1
chr1    GTF2GFF gene    11874   14408   .   +   .   ID=DDX11L1;Note=unknown;Name=DDX11L1
chr1    GTF2GFF exon    11874   12227   .   +   .   Parent=NR_046018_1
chr1    GTF2GFF exon    12613   12721   .   +   .   Parent=NR_046018_1
chr1    GTF2GFF exon    13221   14408   .   +   .   Parent=NR_046018_1
chr1    GTF2GFF gene    14362   29370   .   -   .   ID=WASH7P;Note=unknown;Name=WASH7P
chr1    GTF2GFF exon    14362   14829   .   -   .   Parent=NR_024540
chr1    GTF2GFF exon    14970   15038   .   -   .   Parent=NR_024540
chr1    GTF2GFF exon    15796   15947   .   -   .   Parent=NR_024540
chr1    GTF2GFF exon    16607   16765   .   -   .   Parent=NR_024540
chr1    GTF2GFF exon    16858   17055   .   -   .   Parent=NR_024540
chr1    GTF2GFF exon    17233   17368   .   -   .   Parent=NR_024540
chr1    GTF2GFF exon    17606   17742   .   -   .   Parent=NR_024540
chr1    GTF2GFF exon    17915   18061   .   -   .   Parent=NR_024540
chr1    GTF2GFF exon    18268   18366   .   -   .   Parent=NR_024540
chr1    GTF2GFF exon    24738   24891   .   -   .   Parent=NR_024540
chr1    GTF2GFF exon    29321   29370   .   -   .   Parent=NR_024540
chr1    GTF2GFF gene    34611   36081   .   -   .   ID=FAM138A;Note=unknown;Name=FAM138A
chr1    GTF2GFF exon    34611   35174   .   -   .   Parent=NR_026818
chr1    GTF2GFF exon    35277   35481   .   -   .   Parent=NR_026818

and I want to extract only the rows that that contain "gene" in the 3rd field and re-arrange the 9th field to contain only the ID value (for example, DDX11L1). This is the desired output:

chr1    11874   14408   DDX11L1    .       +
chr1    14362   29370   WASH7P      .       -
chr1    34611   36081   FAM138A    .       -

Using awk I got the desired fields easily:

head -20 genes.gff3 | awk '$3=="gene" {print $1 "\t" $4 "\t" $5 "\t" $9"\t" $6 "\t" $7}'
chr1    11874   14408   ID=DDX11L1;Note=unknown;Name=DDX11L1    .       +
chr1    14362   29370   ID=WASH7P;Note=unknown;Name=WASH7P      .       -
chr1    34611   36081   ID=FAM138A;Note=unknown;Name=FAM138A    .       -

But I am struggling with getting the ID value. I've tried piping it to sed:

head -20 genes.gff3 | awk '$3=="gene" {print $1 "\t" $4 "\t" $5 "\t" $9"\t" $6 "\t" $7}' | sed 's/\(^.+\t\)ID=\(\w+\).+\(\t.+$\)/\1\2\3/g'

and also gsub

head -20 genes.gff3 | awk '$3=="gene" {gsub(/\(^.+\t\)ID=\(\w+\).+\(\t.+$\)/, "\1\2\3", $9); print $1 "\t" $4 "\t" $5 "\t" $9"\t" $6 "\t" $7}' 

But the result is same as using awk alone. How can I extract the ID value? I feel that I am really close to a solution here.

Cheers.

share|improve this question
    
I wrote a Bash function to solve this. I will paste it here if it is fine with you. –  Rany Albeg Wein Apr 26 '13 at 14:23
    
Please do @RanyAlbegWein –  fridaymeetssunday Apr 26 '13 at 14:42
    
is ID always the first var=value? awk '/gene/{sub(/;.*/,"");print}' –  Kevin Apr 26 '13 at 18:15
    
Yes it is @Kevin –  fridaymeetssunday Apr 29 '13 at 8:39

3 Answers 3

up vote 1 down vote accepted

You could split the field and use substr by:

split($9, a, ";")
print substr(a[1], 4)

Awk indexes start at 1.

Another option could be to modify the input field separator (FS). FS is space, " ", by default – which also has the special effect of ignoring leading and trailing spaces.

Also, instead of using print $1, \t, ... or the printf variant one could set OFS to tab.


Examples:

Modifying FS:

awk -F" +|;|=" '

$3 == "gene" {
    printf("%s\t%s\t%s\t%s\t%s\t%s\t\n",
    $1, $4, $5, $10, $6, $7);
}
' data.file

Using split:

awk '
$3 == "gene" {
    split($9, a, ";")
    printf("%s\t%s\t%s\t%s\t%s\t%s\t\n",
    $1, $4, $5, substr(a[1], 3), $6, $7);
}
' data.file

OFS and FS:

Output Field Separator (OFS) as tab, and alternative FS inside awk. Also updated FS to include tab:

awk '
BEGIN {
    FS="[ \t]+|;|="
    OFS="\t"
}
$3 == "gene" {
    print $1, $4, $5, $10, $6, $7
}

' data.file

Also see The Open Group Variables and Special Variables, Examples.

Gawk manual – it usually is noted when things are a gawk extension to awk.

share|improve this answer
    
Getting close but it results in: chr1 11874 14408 . . + ID=DDX11L1 and "ID" should also be removed. –  fridaymeetssunday Apr 26 '13 at 14:01

The field separator of the split function is a regular expression, so you can split on = OR ;. If you know that $9 begins with "ID=", then

awk -v OFS='\t' '
    $3 == "gene" {
        split($9, id, /[=;]/)
        print $1, $4, $5, id[2], $6, $7
    }
' genes.gff3

If "ID=" is not necessarily at the beginning of the field, then there's a little more work to do:

awk -v OFS='\t' '
    $3 == "gene" {
        id = ""
        len = split($9, f, /[=;]/)
        for (i=1; i<len; i++) {
            if (f[i] == "ID") {
                id = f[i+1]
                break
            }
        }
        print $1, $4, $5, id, $6, $7    
    }
' genes.gff3
share|improve this answer
    
Your answer also works, but I've decided to accept Sukminder's because the syntax is simpler. –  fridaymeetssunday Apr 29 '13 at 8:44

This is a Bash solution, as allowed me to publish, despite the explicit request asking to use awk and sed:

show_genes()
{
    local filename="$1"
    while read -ra larr; do
        if [[ ${larr[2]} = gene ]]; then
            larr[8]="${larr[8]%%;*}"
            larr[8]="${larr[8]#ID=}"
            printf '%s\n' "${larr[*]}"
        fi
    done < "$filename"
}

Usage: show_genes /path/to/some/file.txt

Sample output:

[rany$] cat data.txt
romosome  1   249213345   .   .   .   ID=chr1;Name=chr1
chr1    GTF2GFF gene    11874   14408   .   +   .   ID=DDX11L1;Note=unknown;Name=DDX11L1
chr1    GTF2GFF exon    11874   12227   .   +   .   Parent=NR_046018_1
chr1    GTF2GFF exon    12613   12721   .   +   .   Parent=NR_046018_1
chr1    GTF2GFF exon    13221   14408   .   +   .   Parent=NR_046018_1
chr1    GTF2GFF gene    14362   29370   .   -   .   ID=WASH7P;Note=unknown;Name=WASH7P
chr1    GTF2GFF exon    14362   14829   .   -   .   Parent=NR_024540
chr1    GTF2GFF exon    14970   15038   .   -   .   Parent=NR_024540
chr1    GTF2GFF exon    15796   15947   .   -   .   Parent=NR_024540
chr1    GTF2GFF exon    16607   16765   .   -   .   Parent=NR_024540
chr1    GTF2GFF exon    16858   17055   .   -   .   Parent=NR_024540
chr1    GTF2GFF exon    17233   17368   .   -   .   Parent=NR_024540
chr1    GTF2GFF exon    17606   17742   .   -   .   Parent=NR_024540
chr1    GTF2GFF exon    17915   18061   .   -   .   Parent=NR_024540
chr1    GTF2GFF exon    18268   18366   .   -   .   Parent=NR_024540
chr1    GTF2GFF exon    24738   24891   .   -   .   Parent=NR_024540
chr1    GTF2GFF exon    29321   29370   .   -   .   Parent=NR_024540
chr1    GTF2GFF gene    34611   36081   .   -   .   ID=FAM138A;Note=unknown;Name=FAM138A
chr1    GTF2GFF exon    34611   35174   .   -   .   Parent=NR_026818
chr1    GTF2GFF exon    35277   35481   .   -   .   Parent=NR_026818
[rany$] show_genes data.txt
chr1 GTF2GFF gene 11874 14408 . + . DDX11L1
chr1 GTF2GFF gene 14362 29370 . - . WASH7P
chr1 GTF2GFF gene 34611 36081 . - . FAM138A
[rany$]
share|improve this answer
    
Cool thanks. Now I know that this could have also been done in bash. –  fridaymeetssunday Apr 29 '13 at 8:45
    
@fridaymeetssunday You welcome –  Rany Albeg Wein Apr 29 '13 at 11:10

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.