1

I have a file with the following structure:

Locus7625186 GO0004866
Locus7625186 GO0010951
Locus7625186 GO0005615
Locus7625186 GO0016021
Locus7360093 GO0004712
Locus7360093 GO0007093
Locus1507198 GO0044212
Locus1507198 GO0045944
Locus1507198 GO0005634
Locus1507198 GO0036464
Locus1507198 GO0046982

I need to convert it to this:

Locus7625186 GO0004866 GO0010951 GO0005615 GO0016021
Locus7360093 GO0004712 GO0007093
Locus1507198 GO0044212 GO0045944 GO0005634 GO0036464 GO0046982

Note that the number of GOxxxxxxx sharing the same first column match varies.

Improve this question
1
  • I assume the first words always come in groups, like in your example?
    – Philippos
    Sep 18, 2019 at 11:03

5 Answers 5

Reset to default
5

With GNU Datamash

$ datamash -W groupby 1 collapse 2 < file | sed 's/,/ /g'
Locus7625186    GO0004866 GO0010951 GO0005615 GO0016021
Locus7360093    GO0004712 GO0007093
Locus1507198    GO0044212 GO0045944 GO0005634 GO0036464 GO0046982

(you may omit the pipe through sed if you don't mind the default comma separators).

Improve this answer
1
  • Nothing beats a specialized tool +1
    – Philippos
    Sep 18, 2019 at 11:02
2

You may approach this using the GNU sed stream editor:

sed -Ee '
   :a
      $!N
      s/^((\S+)\s.*)\n\2(\s.*)/\1\3/
   ta
   P;D
' file

Results

Locus7625186 GO0004866 GO0010951 GO0005615 GO0016021
Locus7360093 GO0004712 GO0007093
Locus1507198 GO0044212 GO0045944 GO0005634 GO0036464 GO0046982

This can be done using POSIX sed too:

sed -e '
   :a
      $!N
      s/^\(\([^[:space:]]\{1,\}\)[[:space:]].*\)\n\2\([[:space:]].*\)/\1\3/
   ta
   P;D
' file
Improve this answer
1
  • You can do it that way without jump mark and t branching by using the D for looping, making the script more readable. See the explanation of my answer.
    – Philippos
    Sep 18, 2019 at 11:01
1

Another sed approach, compact, portable and bizarre:

sed 'N;/^\(.*\)\( .*\)\(\n\1\)/!P;s//\3\2/;D'
  • It uses the N;P;D approach to always have two lines in pattern space, so it begins with N to append the next line
  • /^\(.*\) .*\n\1/ matches a line starting with some word, a whitespace, another word a newline and the initial word repeated, thus the two lines in the buffer share their first (Locus) word. If this is not the case (!), Print the first line, because it is complete and we can later get rid of it with D
  • But if the line matches, so we have two lines with the same first word and can do a replacement removing the newline and the repeated word. That's why I put two more subgroups in the address pattern for the P, so now I don't have to repeat it, but reuse it by having an empty pattern
  • Now comes the trick: I replace first second\nfirst with \nfirst second, so the pattern space has an empty first line followed by the second line with the first word and all second word we had so far. The following D will now delete the empty first line and continue with the line colleted so far. By including the \n in the \3 match we don't need \n in the replacement, which would not be portable.
Improve this answer
1

Assuming your key values are grouped as in your sample input:

$ awk '
    $1 != prev { printf "%s%s", ors, $1; prev=$1; ors=ORS }
    { printf " %s", $2 }
    END { print "" }
' file
Locus7625186 GO0004866 GO0010951 GO0005615 GO0016021
Locus7360093 GO0004712 GO0007093
Locus1507198 GO0044212 GO0045944 GO0005634 GO0036464 GO0046982

If they aren't then sort the file first.

Improve this answer
-1 
awk '!a[$1]{b[++p]=$1; a[$1]=$2;next} {a[$1]=sprintf("%s%s%s", a[$1], OFS, $2)} END {for (i=1; i<=p; i++) print b[i], a[b[i]]}' file

based in this post how to group rows according to the first field/element

Improve this answer
3
  • For robustness !a[$1] should be !($1 in a) and a[$1]=sprintf("%s%s%s", a[$1], OFS, $2) can be written as just a[$1] = a[$1] OFS $2, the sprintf is adding no value.
    – Ed Morton
    Sep 17, 2019 at 20:32
  • awk '!($1 in a){b[++p]=$1; a[$1]=$2;next} {a[$1] = a[$1] OFS $2} END {for (i=1; i<=p; i++) print b[i], a[b[i]]}' file yes it works and is more efficient. Could you please explain how the whole expression works?
    – Emilio Galarraga
    Sep 17, 2019 at 22:28
  • 1
    !($1 in a) is testing if $1 exists as an index in a[] while !a[$1] is testing if the value that exists in a[] at a[$1] is neither zero nor null and as a side effect creates that index in a. The latter would fail if $2 was 0, for example. a[$1] = a[$1] OFS $2 is just assigning a[$1] the value of the 3 other strings concatenated, exactly as you were doing in the sprintf().
    – Ed Morton
    Sep 17, 2019 at 22:46

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .