Editing a file in place using perl or awk

Question

I have a large tab-delimited input file listed as below

SF_0000000555_RDNAU_58_10293    10873   11041   +       ID=match41;Target=SF_0000000005 99 267 168
SF_0000000555_RDNAU_58_10293    188079  188215  +       ID=match2617;Target=SF_0000000020 3 138 135
SF_0000000555_RDNAU_58_10293    137594  137704  -       ID=match4142;Target=SF_0000000048 16 126 110
SF_0000000555_RDNAU_58_10293    70582   71504   -       ID=match45147;Target=SF_0000000350 8970 9886 916
SF_0000000555_RDNAU_58_10293    100212  101204  -       ID=match45148;Target=SF_0000000350 9584 10597 1013
SF_0000000555_RDNAU_58_10293    101165  101747  -       ID=match45149;Target=SF_0000000350 9005 9581 576
SF_0000000555_RDNAU_58_10293    82434   82891   -       ID=match45150;Target=SF_0000000350 9273 9730 457

I would like the output as given below

SF_0000000555   10873   11041   +       SF_0000000005 99 267 168
SF_0000000555   188079  188215  +       SF_0000000020 3 138 135
SF_0000000555   137594  137704  -       SF_0000000048 16 126 110
SF_0000000555   70582   71504   -       SF_0000000350 8970 9886 916
SF_0000000555   100212  101204  -       SF_0000000350 9584 10597 1013
SF_0000000555   101165  101747  -       SF_0000000350 9005 9581 576
SF_0000000555   82434   82891   -       SF_0000000350 9273 9730 457

Can you please let me know how to edit the file in-place using awk or perl. I have tried using cut command to edit each individual columns and try merging them together using the following command.

awk '{print $1}' |cut -d "_" -f 1-2
awk '{print $5}' |cut -d ";" -f 2- | cut -d "=" -f 2

Thanks in advance.

Answer 1

For editing in place, it may be easier to do something along the lines of

sed -i "s/\t/ /g" file.txt \
&& sed -i "s/ID=match[[:digit:]]\+;Target=//g" test.txt \
&& sed -i "s/_RDNAU_[[:digit:]]\+_[[:digit:]]\+//g" test.txt

Kind of brute force but a hell of a lot easier.

Answer 2

$ awk 'BEGIN{FS=OFS="\t"} {sub(/(_[^_]+){3}$/,"",$1); sub(/.*=/,"",$5)}1' file
SF_0000000555   10873   11041   +       SF_0000000005   99      267     168
SF_0000000555   188079  188215  +       SF_0000000020   3       138     135
SF_0000000555   137594  137704  -       SF_0000000048   16      126     110
SF_0000000555   70582   71504   -       SF_0000000350   8970    9886    916
SF_0000000555   100212  101204  -       SF_0000000350   9584    10597   1013
SF_0000000555   101165  101747  -       SF_0000000350   9005    9581    576
SF_0000000555   82434   82891   -       SF_0000000350   9273    9730    457

With GNU awk you can add -i inplace for "inplace" editing like sed and perl have -i. With any awk you can of course do awk 'script' file > tmp && mv tmp file just like with any other command.

Answer 3

Try this:

awk -v FS='\t' -v OFS='\t' '{
    split($1, a, "_"); $1 = a[1]"_"a[2];
    sub(/.*=/, "", $5);
    print
}' input > output

This assumes that the runs of spaces are actually Tabs in the original file; if that's not the case, you can get rid of the -v.. options.

If the k1=v1;k2=v2;... in the 5th field may be in any order, or if the Target=SF.. may not be the last, something a bit more complicated is needed, but it's hard to second-guess what from the available info.

If you want to edit the file "in-place", you follow that by a rename to the original file:

awk '...' input > tmpfile && mv tmpfile input

Answer 4

In perl:

#!/usr/bin/perl

use feature 'say';

while(<>) {
  chomp;                            # strip the trailing end-of-line character(s).
  my (@F) = split;                  # split the input line into fields

  $F[0] =~ s/(^[^_]*_[^_]*)_.*/$1/; # strip everything from 2nd _ from field 1
  $F[4] =~ s/^ID=[^;]*;Target=//;   # strip from ID= to Target= from field 5

  say join("\t",@F);                # print all the fields separated by a tab.
}

If you want that as a one-liner:

perl -lane '$F[0] =~ s/(^[^_]*_[^_]*)_.*/$1/;
            $F[4] =~ s/^ID=[^;]*;Target=//;
            print join("\t",@F);' input.txt

perl's -l option turns on automatic line-ending processing (chomp). -a turns on awk-like automatic field splitting (to array @F). -a also implicitly turns on -n (which automatically puts a while(<>){ .. } loop around the perl code), but I like to explicitly specify this option. See man perlrun for details.

Note that perl arrays start from 0, not 1.

Sample output (from either stand-alone or one-liner version):

SF_0000000555   10873   11041   +       SF_0000000005   99      267     168
SF_0000000555   188079  188215  +       SF_0000000020   3       138     135
SF_0000000555   137594  137704  -       SF_0000000048   16      126     110
SF_0000000555   70582   71504   -       SF_0000000350   8970    9886    916
SF_0000000555   100212  101204  -       SF_0000000350   9584    10597   1013
SF_0000000555   101165  101747  -       SF_0000000350   9005    9581    576
SF_0000000555   82434   82891   -       SF_0000000350   9273    9730    457

Answer 5

command

awk '{$1=substr($1,1,13);print $0}' filename| sed "s/ID.*=//g"| awk 'OFS ="\t" {print $1,$2,$3,$4,$5,$6,$7,$8}'

output

SF_0000000555   10873   11041   +   SF_0000000005   99  267 168
SF_0000000555   188079  188215  +   SF_0000000020   3   138 135
SF_0000000555   137594  137704  -   SF_0000000048   16  126 110
SF_0000000555   70582   71504   -   SF_0000000350   8970    9886    916
SF_0000000555   100212  101204  -   SF_0000000350   9584    10597   1013
SF_0000000555   101165  101747  -   SF_0000000350   9005    9581    576
SF_0000000555   82434   82891   -   SF_0000000350   9273    9730    457