Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have data like this:

chr1    134901  139379  -   "ENSG00000237683.5";
chr1    860260  879955  +   "ENSG00000187634.6";
chr1    861264  866445  -   "ENSG00000268179.1";
chr1    879584  894689  -   "ENSG00000188976.6";
chr1    895967  901095  +   "ENSG00000187961.9";

I generated by parsing a GTF file

I want to remove the "'s and ;'s from column 5 using awk or sed if it possible. The result would look like this:

chr1    134901  139379  -   ENSG00000237683.5
chr1    860260  879955  +   ENSG00000187634.6
chr1    861264  866445  -   ENSG00000268179.1
chr1    879584  894689  -   ENSG00000188976.6
chr1    895967  901095  +   ENSG00000187961.9
share|improve this question
    
you can also use multiple seach and replace statements in sed. sed 's/"//g; s/;//g' filename – jbrahy Jan 14 at 19:55
    
@DigitalTrauma ya, but Dani_l already gave that solution. – jbrahy Jan 14 at 23:59
up vote 5 down vote accepted

Using gsub:

awk '{gsub(/\"|\;/,"")}1' file
chr1    134901  139379  -   ENSG00000237683.5
chr1    860260  879955  +   ENSG00000187634.6
chr1    861264  866445  -   ENSG00000268179.1
chr1    879584  894689  -   ENSG00000188976.6
chr1    895967  901095  +   ENSG00000187961.9

If you want to operate only on the fifth field and preserve any quotes or semicolons in other fields:

awk '{gsub(/\"|\;/,"",$5)}1' file 
share|improve this answer
    
This worked excellently. Will accept answer when allowed! Thank you. – System Jan 14 at 19:48
1  
This would remove from all columns, not just 5th, no? – Dani_l Jan 14 at 19:55
    
This is what I thought initally, but after using the code it seemed to keep all columns. – System Jan 14 at 19:57
    
@Dani_l Yes, it can be refined to operate only on the fifth field, but that was not a requirement... – jasonwryan Jan 14 at 19:57
    
Sorry I must have not made it clear, I DO want to keep all columns. This is why it is marked as the answer. – System Jan 14 at 19:58

Using sed to remove all instances of '";': sed -i 's/[";]//g' file

To only remove from 5th column sed is probably not the best option.

share|improve this answer

If your data is formatted exactly as shown (i.e. no other " or ; in other columns that need to be preserved), then you can simply use tr to remove these characters:

tr -d '";' < input.txt > output.txt
share|improve this answer

I know the original post asked for sed or awk but if you want to remove the " and ; from only the fifth column I'd use regex and php. There's probably a way to do this in AWK but I like to use the easiest tools.

<?php

foreach(file($argv[1]) as $line){

    $matches = array();
    preg_match('/^(\w+)\s+(\d+)\s+(\d+)\s+(\-|\+)\s+"(\w+.\d)"\;/',$line,$matches);
    $matched_line = array_shift($matches); // remove the first element
    vprintf("%s\t%s\t%s\t%s\t%s\n",$matches);
}

this would output this

$ php /tmp/preg_replace.php /tmp/data
chr1    134901  139379  -   ENSG00000237683.5
chr1    860260  879955  +   ENSG00000187634.6
chr1    861264  866445  -   ENSG00000268179.1
chr1    879584  894689  -   ENSG00000188976.6
chr1    895967  901095  +   ENSG00000187961.9
share|improve this answer
1  
I'm not sure how this satisfies the "easiest tools" criteria; just the amont of typing alone... – jasonwryan Jan 14 at 20:17
    
I prefer php to awk and sed and this is the only answer that actually does what the original post requested by removing " and ; from only the fifth column. Give me that point back. – jbrahy Jan 14 at 20:18
    
I wasn't the downvoter, and no, my edited answer also only operates on the fifth field (and has other advantages besides brevity)... – jasonwryan Jan 14 at 20:24
    
ah, ok. I didn't see the edited version. $5 is definitely less typing. For me PHP code is easier so I provided a solution I thought would help someone. – jbrahy Jan 14 at 20:25
    
Fair enough, it is always good to see solutions using different approaches... – jasonwryan Jan 14 at 20:46

A sed solution that makes sure we're only fiddling around with the fifth column:

sed -r 's/^(([^ ]+ +){4})"([^"]+)";$/\1\3/' infile
chr1    134901  139379  -   ENSG00000237683.5
chr1    860260  879955  +   ENSG00000187634.6
chr1    861264  866445  -   ENSG00000268179.1
chr1    879584  894689  -   ENSG00000188976.6
chr1    895967  901095  +   ENSG00000187961.9

Works also without ERE (-r), but requires a lot more backslashes. + is GNU extension and can be replaced by {1,} (or \{1,\} for BRE).

In case the columns aren't space-separated, the spaces can be replaced by the [:space:] POSIX character class to also match tabs.

The regex in detail:

^               # Anchored at start of line
(               # Capture group 1 for first 4 columns
    (           # Capture group 2 for repeat count
        [^ ]+   # 1 or more non-spaces
         +      # 1 or more spaces
    ){4}        # 4 times "word plus spaces" (columns)
)               # End capture group 1
"               # Column 5 starts with double quote (not captured)
(               # Capture group 3 for column 5
    [^"]+       # One or more non-quote characters
)               # End capture group 3
";              # Quote and semicolon at end of column 5
$
share|improve this answer

If every line has fixed length (as in the example) than

cut -c1-28,30-46 INFILE

will work.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.