4

I am struggled myself to make a case sensitive replacement in a text file. Please find below a segment of my sed file that I am running as sed -f file.sed < input.txt > output.txt

 s/\<code_229633_13\>/R77_08349T0/
 s/\<code_229633_138\>/R77_09738T0/
 s/\<code_230519_10\>/R77_04813T0/
 s/\<code_230519_1\>/R77_13591T0/
 s/\<code_230519_13\>/R77_05463T0/
 up to line 14521....

The code is working great but I have also cases where I have 2 or more TARGET ids (code_010512_23 and code_299097_0) ovelapping the same REPLACEMENT id (R77_14520T0) and I would like to have as output something like R77_14520T0.a and R77_14520T0.b (lines 1 and 2 below)

s/code_010512_23/R77_14520T0/ --> R77_14520T0.a
s/code_299097_0/R77_14520T0/ --> R77_14520T0.b

Furthermore, a more complex but similar case is when i have the following input file (input2.txt file):

  ID=gene09464;Name=code_229633_13;isoforms=1           
  ID=mRNA10661;Parent=gene09464;Name=code_229633_13         
  ID=exon26192;Parent=mRNA10661;Name=code_229633_13;Target=R77_08349T0  1   1093    +
  ID=exon26193;Parent=mRNA10661;Name=code_229633_13;Target=R77_08349T0  1094    1873    +

  ID=gene09491;Name=code_229633_138;isoforms=1          
  ID=mRNA10690;Parent=gene09491;Name=code_229633_138            
  ID=exon26252;Parent=mRNA10690;Name=code_229633_138;Target=R77_09738T0 1   411 +

  ID=gene09513;Name=code_230519_10;isoforms=1           
  ID=mRNA10715;Parent=gene09513;Name=code_230519_10         
  ID=exon26311;Parent=mRNA10715;Name=code_230519_10;Target=R77_04813T0  1   59  +
  ID=exon26312;Parent=mRNA10715;Name=code_230519_10;Target=R77_04813T0  60  186 +

  ID=gene09511;Name=code_230519_1;isoforms=1            
  ID=mRNA10713;Parent=gene09511;Name=code_230519_1          
  ID=exon26308;Parent=mRNA10713;Name=code_230519_1;Target=R77_13591T0   1   1075    +
  ID=exon26309;Parent=mRNA10713;Name=code_230519_1;Target=R77_13591T0   1076    1128    +

  ID=gene09514;Name=code_230519_13;isoforms=1           
  ID=mRNA10716;Parent=gene09514;Name=code_230519_13         
  ID=exon26316;Parent=mRNA10716;Name=code_230519_13;Target=R77_05463T0  1   219 +

  ID=gene00865;Name=code_010512_23;isoforms=1           
  ID=mRNA00979;Parent=gene00865;Name=code_010512_23         
  ID=exon02477;Parent=mRNA00979;Name=code_010512_23;Target=R77_14520T0  1   143 +

  ID=gene14561;Name=code_299097_0;isoforms=2            
  ID=mRNA16419;Parent=gene14561;Name=code_299097_0          
  ID=exon39828;Parent=mRNA16419;Name=code_299097_0;Target=R77_14520T0   144 193 +
  ID=mRNA16420;Parent=gene14561;Name=code_299097_0          
  ID=exon39828;Parent=mRNA16420;Name=code_299097_0;Target=R77_15554T0   408 457 +

and I need to apply the replacements with the same as the previous way only on the lines which contain the word "isoforms", in other words in lines 1,6,10, 15,20, 24 and 28 and nowhere else in the text. The format of this input file would be exactly as depicted with blank lines among the "isoforms" lines.

My desired output

 ID=gene09464;Name=R77_08349T0;isoforms=1           
 ID=mRNA10661;Parent=gene09464;Name=code_229633_13          
 ID=exon26192;Parent=mRNA10661;Name=code_229633_13;Target=R77_08349T0   1   1093    +
 ID=exon26193;Parent=mRNA10661;Name=code_229633_13;Target=R77_08349T0   1094    1873    +
 ID=exon26194;Parent=mRNA10661;Name=code_229633_13;Target=R77_08349T0   1874    4065    +

 ID=gene09491;Name=R77_09738T0;isoforms=1           
 ID=mRNA10690;Parent=gene09491;Name=code_229633_138         
 ID=exon26252;Parent=mRNA10690;Name=code_229633_138;Target=R77_09738T0  1   411 +

 ID=gene09513;Name=Target=R77_04813T0;isoforms=1            
 ID=mRNA10715;Parent=gene09513;Name=code_230519_10          
 ID=exon26311;Parent=mRNA10715;Name=code_230519_10;Target=R77_04813T0   1   59  +
 ID=exon26312;Parent=mRNA10715;Name=code_230519_10;Target=R77_04813T0   60  186 +
 ID=exon26313;Parent=mRNA10715;Name=code_230519_10;Target=R77_04813T0   187 678 +
 ID=exon26314;Parent=mRNA10715;Name=code_230519_10;Target=R77_04813T0   679 1399    +
 ID=exon26315;Parent=mRNA10715;Name=code_230519_10;Target=R77_04813T0   1400    1402    +

 ID=gene09511;Name=R77_13591T0;isoforms=1           
 ID=mRNA10713;Parent=gene09511;Name=code_230519_1           
 ID=exon26308;Parent=mRNA10713;Name=code_230519_1;Target=R77_13591T0    1   1075    +
 ID=exon26309;Parent=mRNA10713;Name=code_230519_1;Target=R77_13591T0    1076    1128    +

 ID=gene09514;Name=R77_05463T0;isoforms=1           
 ID=mRNA10716;Parent=gene09514;Name=code_230519_13          
 ID=exon26316;Parent=mRNA10716;Name=code_230519_13;Target=R77_05463T0   1   219 +

 ID=gene00865;Name=R77_14520T0.a;isoforms=1         
 ID=mRNA00979;Parent=gene00865;Name=code_010512_23          
 ID=exon02477;Parent=mRNA00979;Name=code_010512_23;Target=R77_14520T0   1   143 +

 ID=gene14561;Name=R77_14520T0.b;isoforms=2         
 ID=mRNA16419;Parent=gene14561;Name=code_299097_0           
 ID=exon39828;Parent=mRNA16419;Name=code_299097_0;Target=R77_14520T0    144 193 +
 ID=mRNA16420;Parent=gene14561;Name=code_299097_0           
 ID=exon39828;Parent=mRNA16420;Name=code_299097_0;Target=R77_15554T0    408 457 +
  • Please show us your input and desired output. It is very hard to understand your description of the problem and an example would clarify. Do you want the text line 1 and --> in your output for example? – terdon Mar 3 '14 at 15:50
  • Thank you for your reply.I have modified my initial question adding an example of my input and desired output file. I hope now makes more sense!! – user61677 Mar 3 '14 at 17:11
  • Do your files actually contain the text line 1 etc? And the space at the beginning of each line? – terdon Mar 3 '14 at 17:12
  • No both my input and output files does not contain the text "line 1" etc, I have just added them for helping understand the lines numbering. There is no space at the beginning of each line.For example first line starts with "isoforms=1" – user61677 Mar 3 '14 at 17:25
  • 1
    Please always show your exact data, you never know what can affect the parsing. – terdon Mar 3 '14 at 17:49
4

You can't really do this kind of thing with sed, it's just a text stream editor. Try this Perl scriptlet:

#!/usr/bin/env perl 

## Set the record separator to \n\n to
## read multiple lines as a single record
$/="\n\n";
## This array will contain all lines of the file
my @lines=<>;

## The list of suffixes
@suffix=(a..z); 

## For each line of the input file
foreach (@lines) {
    ## If the current line (lines are now the actual multiline records
    ## because we set $/ to consecutive newlines) is one we are interested in.
    if (/isoforms.*?Target=(\S+)/s){
    ## Keep a list of seen targets
    $seen{$1}++;
    }

}
## Now that we have processed the entire file
## go back and print each line.
foreach (@lines) {

    ## If this line is one of the ones we're interested in
    if(/Name=(.+?);.*?isoforms=.*?Target=(\S+)/s){
    $name=$1; $target=$2;
    ## This is needed so we can know whether
    ## how many times we've seen this target so far.
    $newseen{$target}++;
    ## If this target exists more than once in the input file
    if ($seen{$target}>1) {
        ## Use the %newseen hash to choose the right letter.
        ## The -1 is needed because the first element of an
        ## array is 0, not 1.
        s/$name/$target.$suffix[$newseen{$target}-1]/;
    }
    else {
        s/$name/$target/;
    }
    }
    print;
}

Save the script above as foo.pl, make it executable (chmod a+x foo.pl) and run on your input file:

./foo.pl input.txt > output.txt
  • Thanks for the code!!Please see my fully reply to my initial answer. – user61677 Mar 3 '14 at 22:19
  • Dear @terdon once again thanks. It works now!!!The problem solved regarding the initial input file!!Could you please indicate me something to apply these replacements only on the lines which contain the word "isoforms" or "gene", in other words in "initial" lines 1,6,10, 15,20, 24 and 28 and nowhere else in the input2.txt file. It is so important...I think in the end of your "if" statement (line 35) there could be start a new loop and apply the replacements in the lines that I am interested. – user61677 Mar 4 '14 at 9:20
  • I remarked that your updated script now performs these replacements: isoforms=1 Name=R77_14520T0.a Target=R77_14520T0 isoforms=2 Name=R77_14520T0.b Target=R77_14520T0 and unfortunately all the others now remain unchanged.. – user61677 Mar 4 '14 at 18:00
  • @user61677 I'm not sure what you mean anymore. No lines contain both isoforms and Target, those occur on different lines. Do you mean you want the substitution only on records that have multiple isoforms? It might be easier if you just ping me in Unix & Linux Chat and explain there. – terdon Mar 4 '14 at 18:10
  • @user61677 also, please explain what you would like Name to be replaced by. The gene's name? The transcript's? – terdon Mar 4 '14 at 18:21
0

Forget what I wrote before, I thought you were moving files. I think I understand it better now:

sed -r '/isoforms/{\
    N;N;s/(.*Name=)[^;]*(.*Name=)\S*\
    (.*Name=.*Target=)(\S*)(.*)/\
        \1\4\2\4\3\4\5/}'

I ran it and it works for me to replace the first two occurrences of "Name=code[0-9_]*" on any set of three lines beginning with one containing "isoforms" with the next occurring non-whitespace sequence $VALUE of the "Target=$VALUE" key pair.

I only used a sample of 20 or so lines; so probably it needs tuning. Also, I'm not certain I understand the ".[a|b]" suffix problem you mention, though I am certain I didn't address it yet. I am fairly confident that whatever it is can be handled with a branch test or two. I'll try to explain what the code means:

  • /isoforms/ sed first watches the stream for any line containing this string.

  • { When found, it begins to execute the code block located within the curly braces. }

  • N;N; sed immediately reads the next two lines of input into its pattern space so it can operate on three lines at once. If you used an "l" command at this point to print the pattern space you'd see something like:

    isoforms line \n gene line \n target line

  • s/search/replace/ the s command instructs sed to perform a search and replace function on the current pattern space.

  • (regex) each parens pair in the regular expression search field matches a back reference in the replacement field. Some shorthand I used:

    • [^;]* the next occurring longest sequence of characters not including a semicolon.

    • \S* the next occurring longest sequence of characters not including whitespace.

  • \NUM sed increments the replacement backreferences in the order they're defined in the search field. A short breakdown of this specific search:

    • \1 sed searches for the "Name=" string and, finding three in the pattern space and three specified in the regex, matches everything up to and including its first result.

    • DISCARDED sed discards the longest sequence of characters it can match following \1 without including a semicolon.

    • \2 Just like \1 but for the second "Name=" result.

    • DISCARDED sed discards the next occurring sequence of characters until it encounters whitespace.

    • \3 sed matches "Name=" and everything else up to and including "Target="

    • \4 sed matches the next sequence of characters until it encounters whitespace

    • \5 sed matches what remains of line 3

  • To put it all together, I just replaced the DISCARDED sections with \4 like this:

    \1\4\2\4\3\4\5

INPUT:

ID=gene09491;Name=code_229633_138;isoforms=1
ID=mRNA10690;Parent=gene09491;Name=code_229633_138
ID=exon26252;Parent=mRNA10690;Name=code_229633_138;Target=R77_09738T0 1 411 +

ID=gene09513;Name=code_230519_10;isoforms=1
ID=mRNA10715;Parent=gene09513;Name=code_230519_10
ID=exon26311;Parent=mRNA10715;Name=code_230519_10;Target=R77_04813T0 1 59 + ID=exon26312;Parent=mRNA10715;Name=code_230519_10;Target=R77_04813T0 60 186 +

OUTPUT:

ID=gene09491;Name=R77_09738T0;isoforms=1
ID=mRNA10690;Parent=gene09491;Name=R77_09738T0
ID=exon26252;Parent=mRNA10690;Name=code_229633_138;Target=R77_09738T0 1 411 +

ID=gene09513;Name=R77_04813T0;isoforms=1
ID=mRNA10715;Parent=gene09513;Name=R77_04813T0
ID=exon26311;Parent=mRNA10715;Name=code_230519_10;Target=R77_04813T0 1 59 +
ID=exon26312;Parent=mRNA10715;Name=code_230519_10;Target=R77_04813T0 60 186 +

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.