1

I have a large tab-delimited input file listed as below

SF_0000000555_RDNAU_58_10293    10873   11041   +       ID=match41;Target=SF_0000000005 99 267 168
SF_0000000555_RDNAU_58_10293    188079  188215  +       ID=match2617;Target=SF_0000000020 3 138 135
SF_0000000555_RDNAU_58_10293    137594  137704  -       ID=match4142;Target=SF_0000000048 16 126 110
SF_0000000555_RDNAU_58_10293    70582   71504   -       ID=match45147;Target=SF_0000000350 8970 9886 916
SF_0000000555_RDNAU_58_10293    100212  101204  -       ID=match45148;Target=SF_0000000350 9584 10597 1013
SF_0000000555_RDNAU_58_10293    101165  101747  -       ID=match45149;Target=SF_0000000350 9005 9581 576
SF_0000000555_RDNAU_58_10293    82434   82891   -       ID=match45150;Target=SF_0000000350 9273 9730 457

I would like the output as given below

SF_0000000555   10873   11041   +       SF_0000000005 99 267 168
SF_0000000555   188079  188215  +       SF_0000000020 3 138 135
SF_0000000555   137594  137704  -       SF_0000000048 16 126 110
SF_0000000555   70582   71504   -       SF_0000000350 8970 9886 916
SF_0000000555   100212  101204  -       SF_0000000350 9584 10597 1013
SF_0000000555   101165  101747  -       SF_0000000350 9005 9581 576
SF_0000000555   82434   82891   -       SF_0000000350 9273 9730 457

Can you please let me know how to edit the file in-place using awk or perl. I have tried using cut command to edit each individual columns and try merging them together using the following command.

awk '{print $1}' |cut -d "_" -f 1-2
awk '{print $5}' |cut -d ";" -f 2- | cut -d "=" -f 2

Thanks in advance.

1
  • 3
    Editing "in-place" usually means replacing the input file with the edited data. Is this what you mean as well? I just noticed you also say that you expect some specific "output", which seems to contradict "in-place editing".
    – Kusalananda
    Aug 6, 2019 at 20:45

4 Answers 4

1

For editing in place, it may be easier to do something along the lines of

sed -i "s/\t/ /g" file.txt \
&& sed -i "s/ID=match[[:digit:]]\+;Target=//g" test.txt \
&& sed -i "s/_RDNAU_[[:digit:]]\+_[[:digit:]]\+//g" test.txt

Kind of brute force but a hell of a lot easier.

3
  • Note that some fields in the data has been edited between input and output. I suspect that they want tab-delimited output.
    – Kusalananda
    Aug 6, 2019 at 20:48
  • Ah, didn't notice. Using sed would still apply though, you'd just need a second application of it, I'll see if I can draft one up. Aug 6, 2019 at 20:49
  • 1
    If you want, you can sed -i -e foo -e bar instead of sed -i foo && sed -i bar :) It might make a difference on a large file, like OP asked about.
    – user4443
    Aug 6, 2019 at 21:50
1
$ awk 'BEGIN{FS=OFS="\t"} {sub(/(_[^_]+){3}$/,"",$1); sub(/.*=/,"",$5)}1' file
SF_0000000555   10873   11041   +       SF_0000000005   99      267     168
SF_0000000555   188079  188215  +       SF_0000000020   3       138     135
SF_0000000555   137594  137704  -       SF_0000000048   16      126     110
SF_0000000555   70582   71504   -       SF_0000000350   8970    9886    916
SF_0000000555   100212  101204  -       SF_0000000350   9584    10597   1013
SF_0000000555   101165  101747  -       SF_0000000350   9005    9581    576
SF_0000000555   82434   82891   -       SF_0000000350   9273    9730    457

With GNU awk you can add -i inplace for "inplace" editing like sed and perl have -i. With any awk you can of course do awk 'script' file > tmp && mv tmp file just like with any other command.

1

Try this:

awk -v FS='\t' -v OFS='\t' '{
    split($1, a, "_"); $1 = a[1]"_"a[2];
    sub(/.*=/, "", $5);
    print
}' input > output

This assumes that the runs of spaces are actually Tabs in the original file; if that's not the case, you can get rid of the -v.. options.

If the k1=v1;k2=v2;... in the 5th field may be in any order, or if the Target=SF.. may not be the last, something a bit more complicated is needed, but it's hard to second-guess what from the available info.

If you want to edit the file "in-place", you follow that by a rename to the original file:

awk '...' input > tmpfile && mv tmpfile input
0
1

In perl:

#!/usr/bin/perl

use feature 'say';

while(<>) {
  chomp;                            # strip the trailing end-of-line character(s).
  my (@F) = split;                  # split the input line into fields

  $F[0] =~ s/(^[^_]*_[^_]*)_.*/$1/; # strip everything from 2nd _ from field 1
  $F[4] =~ s/^ID=[^;]*;Target=//;   # strip from ID= to Target= from field 5

  say join("\t",@F);                # print all the fields separated by a tab.
}

If you want that as a one-liner:

perl -lane '$F[0] =~ s/(^[^_]*_[^_]*)_.*/$1/;
            $F[4] =~ s/^ID=[^;]*;Target=//;
            print join("\t",@F);' input.txt

perl's -l option turns on automatic line-ending processing (chomp). -a turns on awk-like automatic field splitting (to array @F). -a also implicitly turns on -n (which automatically puts a while(<>){ .. } loop around the perl code), but I like to explicitly specify this option. See man perlrun for details.

Note that perl arrays start from 0, not 1.

Sample output (from either stand-alone or one-liner version):

SF_0000000555   10873   11041   +       SF_0000000005   99      267     168
SF_0000000555   188079  188215  +       SF_0000000020   3       138     135
SF_0000000555   137594  137704  -       SF_0000000048   16      126     110
SF_0000000555   70582   71504   -       SF_0000000350   8970    9886    916
SF_0000000555   100212  101204  -       SF_0000000350   9584    10597   1013
SF_0000000555   101165  101747  -       SF_0000000350   9005    9581    576
SF_0000000555   82434   82891   -       SF_0000000350   9273    9730    457
1
  • btw, the say feature requires perl >= version 5.10, which was released in Dec 2007. For older versions of perl, just use print join("\t",@F), "\n"; - say is just a convenience feature so you don't have to explicitly put a newline on the end of everything you print (i.e. to reduce the amount of typing required)
    – cas
    Aug 7, 2019 at 2:04

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .