1

I have a large tab-delimited input file listed as below

SF_0000000555_RDNAU_58_10293    10873   11041   +       ID=match41;Target=SF_0000000005 99 267 168
SF_0000000555_RDNAU_58_10293    188079  188215  +       ID=match2617;Target=SF_0000000020 3 138 135
SF_0000000555_RDNAU_58_10293    137594  137704  -       ID=match4142;Target=SF_0000000048 16 126 110
SF_0000000555_RDNAU_58_10293    70582   71504   -       ID=match45147;Target=SF_0000000350 8970 9886 916
SF_0000000555_RDNAU_58_10293    100212  101204  -       ID=match45148;Target=SF_0000000350 9584 10597 1013
SF_0000000555_RDNAU_58_10293    101165  101747  -       ID=match45149;Target=SF_0000000350 9005 9581 576
SF_0000000555_RDNAU_58_10293    82434   82891   -       ID=match45150;Target=SF_0000000350 9273 9730 457

I would like the output as given below

SF_0000000555   10873   11041   +       SF_0000000005 99 267 168
SF_0000000555   188079  188215  +       SF_0000000020 3 138 135
SF_0000000555   137594  137704  -       SF_0000000048 16 126 110
SF_0000000555   70582   71504   -       SF_0000000350 8970 9886 916
SF_0000000555   100212  101204  -       SF_0000000350 9584 10597 1013
SF_0000000555   101165  101747  -       SF_0000000350 9005 9581 576
SF_0000000555   82434   82891   -       SF_0000000350 9273 9730 457

Can you please let me know how to edit the file in-place using awk or perl. I have tried using cut command to edit each individual columns and try merging them together using the following command.

awk '{print $1}' |cut -d "_" -f 1-2
awk '{print $5}' |cut -d ";" -f 2- | cut -d "=" -f 2

Thanks in advance.

  • 3
    Editing "in-place" usually means replacing the input file with the edited data. Is this what you mean as well? I just noticed you also say that you expect some specific "output", which seems to contradict "in-place editing". – Kusalananda Aug 6 at 20:45
1

For editing in place, it may be easier to do something along the lines of

sed -i "s/\t/ /g" file.txt \
&& sed -i "s/ID=match[[:digit:]]\+;Target=//g" test.txt \
&& sed -i "s/_RDNAU_[[:digit:]]\+_[[:digit:]]\+//g" test.txt

Kind of brute force but a hell of a lot easier.

  • Note that some fields in the data has been edited between input and output. I suspect that they want tab-delimited output. – Kusalananda Aug 6 at 20:48
  • Ah, didn't notice. Using sed would still apply though, you'd just need a second application of it, I'll see if I can draft one up. – Zach Sanchez Aug 6 at 20:49
  • 1
    If you want, you can sed -i -e foo -e bar instead of sed -i foo && sed -i bar :) It might make a difference on a large file, like OP asked about. – user4443 Aug 6 at 21:50
1
$ awk 'BEGIN{FS=OFS="\t"} {sub(/(_[^_]+){3}$/,"",$1); sub(/.*=/,"",$5)}1' file
SF_0000000555   10873   11041   +       SF_0000000005   99      267     168
SF_0000000555   188079  188215  +       SF_0000000020   3       138     135
SF_0000000555   137594  137704  -       SF_0000000048   16      126     110
SF_0000000555   70582   71504   -       SF_0000000350   8970    9886    916
SF_0000000555   100212  101204  -       SF_0000000350   9584    10597   1013
SF_0000000555   101165  101747  -       SF_0000000350   9005    9581    576
SF_0000000555   82434   82891   -       SF_0000000350   9273    9730    457

With GNU awk you can add -i inplace for "inplace" editing like sed and perl have -i. With any awk you can of course do awk 'script' file > tmp && mv tmp file just like with any other command.

1

Try this:

awk -v FS='\t' -v OFS='\t' '{
    split($1, a, "_"); $1 = a[1]"_"a[2];
    sub(/.*=/, "", $5);
    print
}' input > output

This assumes that the runs of spaces are actually Tabs in the original file; if that's not the case, you can get rid of the -v.. options.

If the k1=v1;k2=v2;... in the 5th field may be in any order, or if the Target=SF.. may not be the last, something a bit more complicated is needed, but it's hard to second-guess what from the available info.

If you want to edit the file "in-place", you follow that by a rename to the original file:

awk '...' input > tmpfile && mv tmpfile input
0

In perl:

#!/usr/bin/perl

use feature 'say';

while(<>) {
  chomp;                            # strip the trailing end-of-line character(s).
  my (@F) = split;                  # split the input line into fields

  $F[0] =~ s/(^[^_]*_[^_]*)_.*/$1/; # strip everything from 2nd _ from field 1
  $F[4] =~ s/^ID=[^;]*;Target=//;   # strip from ID= to Target= from field 5

  say join("\t",@F);                # print all the fields separated by a tab.
}

If you want that as a one-liner:

perl -lane '$F[0] =~ s/(^[^_]*_[^_]*)_.*/$1/;
            $F[4] =~ s/^ID=[^;]*;Target=//;
            print join("\t",@F);' input.txt

perl's -l option turns on automatic line-ending processing (chomp). -a turns on awk-like automatic field splitting (to array @F). -a also implicitly turns on -n (which automatically puts a while(<>){ .. } loop around the perl code), but I like to explicitly specify this option. See man perlrun for details.

Note that perl arrays start from 0, not 1.

Sample output (from either stand-alone or one-liner version):

SF_0000000555   10873   11041   +       SF_0000000005   99      267     168
SF_0000000555   188079  188215  +       SF_0000000020   3       138     135
SF_0000000555   137594  137704  -       SF_0000000048   16      126     110
SF_0000000555   70582   71504   -       SF_0000000350   8970    9886    916
SF_0000000555   100212  101204  -       SF_0000000350   9584    10597   1013
SF_0000000555   101165  101747  -       SF_0000000350   9005    9581    576
SF_0000000555   82434   82891   -       SF_0000000350   9273    9730    457
  • btw, the say feature requires perl >= version 5.10, which was released in Dec 2007. For older versions of perl, just use print join("\t",@F), "\n"; - say is just a convenience feature so you don't have to explicitly put a newline on the end of everything you print (i.e. to reduce the amount of typing required) – cas Aug 7 at 2:04
-1

command

awk '{$1=substr($1,1,13);print $0}' filename| sed "s/ID.*=//g"| awk 'OFS ="\t" {print $1,$2,$3,$4,$5,$6,$7,$8}'

output

SF_0000000555   10873   11041   +   SF_0000000005   99  267 168
SF_0000000555   188079  188215  +   SF_0000000020   3   138 135
SF_0000000555   137594  137704  -   SF_0000000048   16  126 110
SF_0000000555   70582   71504   -   SF_0000000350   8970    9886    916
SF_0000000555   100212  101204  -   SF_0000000350   9584    10597   1013
SF_0000000555   101165  101747  -   SF_0000000350   9005    9581    576
SF_0000000555   82434   82891   -   SF_0000000350   9273    9730    457

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.