0

I have a file like this:

$ cat file
    rep1 rep2
g1001_INpfu_DN44908_c3_g1 17.85 19.95
g10042/1330/2846_INpfu_DN43979_c0_g3 34.07 29.19
g1077/1457/278/278_INpfu_PRJNA287145_DN42983_c0_g1 20.69 21.64
g100/100_INpfu_DN43143_c0_g1 52.36 33.64
g79/79/79/79_INpfu_DN45068_c4_g1 58.83 74.58

The text before the first "_" is group number(s), like

  • gnumber_ ...
  • gnumber1/number2_ ...
  • gnumber1/number2/number3_ ...

etc.  (Spaces are added for illumination.)  Having a slash means multiple groups.  Sometimes there are duplicate group numbers within a line, and I want to remove them.

The expected results should be like:

    rep1 rep2
g1001_INpfu_DN44908_c3_g1 17.85 19.95
g10042/1330/2846_INpfu_DN43979_c0_g3 34.07 29.19
g1077/1457/278_INpfu_PRJNA287145_DN42983_c0_g1 20.69 21.64
g100_INpfu_DN43143_c0_g1 52.36 33.64
g79_INpfu_DN45068_c4_g1 58.83 74.58

In the last three lines, the same group numbers are removed, only keeping the unique group number(s).

2 Answers 2

0

You could try something like this, using sed:

$ sed -e :a -e 's:\([0-9][0-9]*\)/\1:\1:' -e ta file
rep1 rep2
g1001_INpfu_DN44908_c3_g1 17.85 19.95
g10042/1330/2846_INpfu_DN43979_c0_g3 34.07 29.19
g1077/1457/278_INpfu_PRJNA287145_DN42983_c0_g1 20.69 21.64
g100_INpfu_DN43143_c0_g1 52.36 33.64
g79_INpfu_DN45068_c4_g1 58.83 74.58

To handle partial matches such as g512/12/x or g512/5120/x without turning them into g512/x and g5120/x, you could add non-digit anchors either side:

sed -e :a -e 's:\([^0-9]\)\([0-9][0-9]*\)/\2\([^0-9]\):\1\2\3:' -e ta file

or slightly more readably using extended regular expressions

sed -E -e :a -e 's:([^0-9])([0-9]+)/\2([^0-9]):\1\2\3:' -e ta file

ex. given

$ cat file
    rep1 rep2
g1001_INpfu_DN44908_c3_g1 17.85 19.95
g10042/1330/2846_INpfu_DN43979_c0_g3 34.07 29.19
g1077/1457/278/278_INpfu_PRJNA287145_DN42983_c0_g1 20.69 21.64
g512/12_INpfu_DN43143_c0_g1 52.36 33.64
g100/100_INpfu_DN43143_c0_g1 52.36 33.64
g512/5120_INpfu_DN43143_c0_g1 52.36 33.64
g79/79/79/79_INpfu_DN45068_c4_g1 58.83 74.58

then

$ sed -E -e :a -e 's:([^0-9])([0-9]+)/\2([^0-9]):\1\2\3:' -e ta file
    rep1 rep2
g1001_INpfu_DN44908_c3_g1 17.85 19.95
g10042/1330/2846_INpfu_DN43979_c0_g3 34.07 29.19
g1077/1457/278_INpfu_PRJNA287145_DN42983_c0_g1 20.69 21.64
g512/12_INpfu_DN43143_c0_g1 52.36 33.64
g100_INpfu_DN43143_c0_g1 52.36 33.64
g512/5120_INpfu_DN43143_c0_g1 52.36 33.64
g79_INpfu_DN45068_c4_g1 58.83 74.58
6
  • Thank you so much! It works!!
    – ruby Sheng
    May 2, 2020 at 3:09
  • That turns g512/12/x into g512/x though May 2, 2020 at 7:42
  • @StéphaneChazelas thanks for catching that May 2, 2020 at 10:49
  • That still turns g512/5120/x into g5120/x. May 3, 2020 at 0:33
  • @G-ManSays'ReinstateMonica' yeah I'd like to delete this answer in light of the obviously better one but unfortunately that doesn't appear to be possible now that it's been accepted May 3, 2020 at 0:39
0

With perl:

perl -pe 's{^g(?:\d+/)*?(\d+)\K(?:/\1)+(?!\d)}{}' < your-file

It is strict in that it will remove the first sequence of repeating numbers only in the leading g/number/number[/number...]/number part and only if that part is not followed by a digit.

With sed, you could do the same with:

sed '
  \|^\(g\([[:digit:]]\{1,\}/\)*[[:digit:]]\{1,\}\).*| {
    h; # save a copy of original line
    s||\1:|; # remove all but the leading g/x/y/z
    s|\([g/]\)\([[:digit:]]\{1,\}\)\(/\2\)\{1,\}\([^[:digit:]]\)|\1\2\4|
    G; # append saved copy
    s|:\ng\([[:digit:]]\{1,\}/\)*[[:digit:]]\{1,\}||; # remove excess
  }' < your-file
2
  • @Isaac, yes the OP doesn't say only the consecutive repeats have to be trimmed, I possibly incorrectly derived that from his sample where it was the case. They don't say either that only one set of repeated numbers have to be trimmed. But the text of my answer does say what it does. May 2, 2020 at 19:59
  • @Isaac. I don't find \# particularly less confusing than\|. I find a vertical bar works quite well as a delimiter, but feel free to use # if you prefer. I'll grant you that | doesn't work in vim's s command or could be confusing when using EREs. May 2, 2020 at 20:00

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .