I have code I normally run in R, but the file is too big, so I am trying to run the same commands in awk.
I am trying to group values in columns by an ID column (or Gene
column in my case).
My data looks like:
Gene col1 col2 col3
ACE 1 0.4 BP
ACE 2 0.5 DP
RPP-I.1 1 0.01 BP
NOS2 -0.1 0.2 DP
NOS2 1.4 2.5 SP
NOS2 1 1 BP
I want to group it by Gene
to look like:
Gene col1 col2 col3
ACE 1, 2 0.4, 0.5 BP, DP
RPP-I.1 1 0.01 BP
NOS2 -0.1, 1.4, 1 0.2, 2.5, 1 BP, SP, DP
My real data is 14.8GB with ~200 columns and 24972316 lines, I was originally trying to use R's data.table but this is giving a bus error with trying to read in the file.
Is there a way I can try this with awk?
Gene
column, and it is a character class with no numeric values - I'll change my example genes to real example to be more clear, sorry about thatGene3
+BP
? Do they need to be counted somehow in the output, or is it sufficient to show them once?