4

I am very new to unix as well as coding in general and have spent a long time trying to figure out this issue with no success. I am sure its a very simple solution but if someone can help point me in the right direction.

I have a Summary.txt file with data with the following format

Name criteria1 2 3 4 5 6 7 8 9 10
Sample12345 w x y z
Sample12345.avg l m n
Sample12345.stdn o p q
Sample98765 w2 x2 y2 z2
Sample98765.avg l2 m2 n2
Sample98765.stdn o2 p2 q2

etc. etc.

I am wanting to combine the three lines of the same sample numbers and only keep the first name:

Name criteria1 2 3 4 5 6 7 8 9 10
Sample12345 w x y z l m n o p q
Sample98765 w2 x2 y2 z2 l2 m2 n2 o2 p2 q2

etc. etc.

I have tried using sed such as (after first making a Summary_temp.txt that contains all Sample12345.avg Sample12345.stdn etc. etc.):

for i in `cat Summary_temp.txt`; do sed 's/\n$i//g' Summary.txt; done gives "-bash: syntax error near unexpected token 'sed'"

I also tried going down the path of

paste -sd ' ' Summary.txt | sed 's/\t/\n/g10; s/\t/@/g; s/\n/\t/g' | sed 's/.\nSample.*.avg//g' | head

I believe the closest I have gotten is:

cat Summary.txt | sed 's/ *| */,/g;/^$/d' | paste -d, - - - | head

However this starts on the 1st line so everything is off by one:

Name criteria1 2 3 4 5 6 7 8 9 10 Sample12345 w x y z Sample12345.avg l m n
Sample12345.stdn o p q Sample98765 w2 x2 y2 z2 Sample98765.avg l2 m2 n2
Sample98765.stdn o2 p2 q2 ETC ETC

I have tried head -1 Summary.txt > Summary_temp_3.txt so I can then add it back later.

sed '1d' Summary.txt > Summary_temp_4.txt
cat Summary_temp_4.txt | sed 's/ *| */,/g;/^$/d' |
  paste -d, - - - | sed 's/,.*_by_name//g | head

This doesn't show anything at all; I have to eventually ctrl-C, so assume it is wrong because it shouldn't take that long. Please help. I know I am very much a beginner and this may be an easily solvable dumb question but I don't know what else to try.

0

4 Answers 4

10

It's not especially elegant, but for the second through last line (2,$) pull the Next line and the Next line into pattern space, then globally replace any non-blank sequences following a newline:

sed '
2,${
N
N
s/\n[^[:blank:]]*//g
}
' Summary.txt
Name criteria1 2 3 4 5 6 7 8 9 10
Sample12345 w x y z l m n o p q
Sample98765 w2 x2 y2 z2 l2 m2 n2 o2 p2 q2
0
6

A paste + sed approach could be (assuming the file doesn't contain , characters):

$ paste -sd '\n,,' your-file | sed 's/,[^[:blank:]]*//g'
Name criteria1 2 3 4 5 6 7 8 9 10
Sample12345 w x y z l m n o p q
Sample98765 w2 x2 y2 z2 l2 m2 n2 o2 p2 q2
0
4

If your input data is consistent, i.e. every three lines share first field of the first line in the group, then you simply need an awk command -- something like this:

awk '
    NR==1{print;next} 
    NR%3==0{$1="";printf $0} 
    NR%3==1{$1="";printf $0;print ""} 
    NR%3==2{printf $0} 
   ' infile

Result:

Name criteria1 2 3 4 5 6 7 8 9 10
Sample12345 w x y z l m n o p q
Sample98765 w2 x2 y2 z2 l2 m2 n2 o2 p2 q2
0
2

Using Raku (formerly known as Perl_6)

What you're describing might best be handled by a hash, since the keys are maintained unique. Additional key-value pairs get added to the hash as new pairs if no such key exists, otherwise the values are appended to a pre-existing key of the same name. As an simple example:

~$ raku -ne 'BEGIN my %h; %h.append: .split(" ", 2); END .put for %h.sort;' file

The above code handles a simple first-column "key", with the remainder of the line becoming the "value". For the more complicated dataset given by the OP, you need to eliminate the .stdn and .avg extensions:

~$ raku -ne 'BEGIN my %h;  \
             .split(/ \. <alpha>+ | \s+ /, :skip-empty) andthen  \
             %h.append(.[0] => .[1..*]);   \     
             END .put for %h.sort;'  file

Explaining more fully: Raku code is run using the -ne non-autoprinting linewise flags. Before looping through lines, we BEGIN by declaring a hash, %h. Lines are then split on either \. <alpha>+ (e.g. ".avg" or ".stdn"), or \s+ on whitespace. Once split we reload $_ with andthen, andthen append a key-value pair to %h hash. The => "fat-arrow" syntax denotes/creates a key-value pair, as in .[0] => .[1..*]. The first column (index = 0) becomes the key, while the remaining (index = 1..* ) elements become the values.

Sample Input:

Name criteria1 2 3 4 5 6 7 8 9 10
Sample12345 w x y z
Sample12345.avg l m n
Sample12345.stdn o p q
Sample98765 w2 x2 y2 z2
Sample98765.avg l2 m2 n2
Sample98765.stdn o2 p2 q2

Sample Output (TAB between key and value):

Name    criteria1 2 3 4 5 6 7 8 9 10
Sample12345 w x y z l m n o p q
Sample98765 w2 x2 y2 z2 l2 m2 n2 o2 p2 q2

Finally, if you want to use the output for a CSV file, you can join all elements on comma (including individual values). Full answer below:

~$ raku -ne 'BEGIN my %h;  \
             .split(/ \. <alpha>+ | \s+ /, :skip-empty) andthen  \
             %h.append(.[0] => .[1..*]);   \
             END put .keys ~ "," ~ .values.map: *.join(",") for %h.sort;'  file
Sample98765,w2,x2,y2,z2,l2,m2,n2,o2,p2,q2
Sample12345,w,x,y,z,l,m,n,o,p,q
Name,criteria1,2,3,4,5,6,7,8,9,10

As a double-check on the code, changing:

.values.map: *.join(",") to .values.map: *.elems

Returns the number of value columns, 10 for each key:

Name,10
Sample12345,10
Sample98765,10

https://docs.raku.org/language/hashmap
https://raku.org

0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .