Extract information from a text file

Question

I have a text file, i want to search for tags such as the following:

<category="SpecificDisease">Type II human complement C2 deficiency</category>
<category="Modifier">Huntington disease</category>
<category="CompositeMention">hereditary breast and ovarian cancer</category>
<category="DiseaseClass">myopathy</category>

and produce the following and write them to a new text file.

Type II human complement C2 deficiency
Huntington disease
hereditary breast and ovarian cancer
myopathy

Answer 1

That looks like a kind of XML or similar markup language file. Such files should not be parsed by simple regular expressions for fear of waking TO͇̹̺ͅƝ̴ȳ̳ TH̘Ë͖́̉ ͠P̯͍̭O̚​N̐Y̡. You should use a parser specific for that markup and your favorite scripting language instead.

That looks like OMIM or HPO data in which case you should be able to get simple text files and simplify things. If you can't and really really need to parse this file, you could do this in perl:

perl -lne '/<.*?>([^<>]+)/ && print $1' foo.txt

However, this will break if you have multiple tags per line, or if your tag's contents can span multiple lines or if the tag's data can contain > or <. If all your info is always between <category="whatever">blah blah</category>, you can get everything more robustly (including multi line tag contents and embedded < or >):

#!/usr/bin/env perl

## Set the start and end tags
$end="</category>"; 
$start="<category=.*?>"; 

## Read through the file line by line
while(<>){
    ## set $a to one if the current line matches $start
    $a=1 if /$start/; 
    ## If the current line matches $start, capture any relevant content.
    ## I am also removing any $start or $end tags if present.
    if(s/($start)*(.+)($end)*/$2/){
    push @lines,$2 if $a==1;
    }  
    ## If the current line matches $end, capture any relevant content,
    ## print what we have saved so far, set $a back to 0 and empty the
    ## @lines array
    if(/$end/){
    map{s/$end//;}@lines; 
    print "@lines\n";
    @lines=(); 
    $a=0
    }; 
}

Save this script as foo.pl or whatever, make it executable and run it on your file:

./foo.pl file.txt

For example:

$ cat file.txt 
<category="SpecificDisease">Type II 
 human complement C2 deficiency</category>
<category="Modifier">Huntington disease</category>
<category="CompositeMention">hereditary breast < and ovarian cancer</category>
<category="DiseaseClass">myopathy > cardiopathy</category>

$ ./foo.pl file.txt 
Type II   human complement C2 deficiency
Huntington disease
hereditary breast < and ovarian cancer
myopathy > cardiopathy

Once more though, I stress that if (which is very likely) your file is more complex than the example above, this will fail and more sophisticated methods will be needed.

Answer 2

You can do this simply in sed:

sed 's/>\([^>]\+\)</\1/' your_file > new_file

to extract the text between > and < on each line. This assumes you have one tag per line.