1

If a data file looks like:

snp200 snp1 snp100 snp32 1 1 0 2 0 0 0 2 2 2 2 1...

how can I break the line when snp.. finish and use them as column names and then divide the rest of digits equally (total number of digits in each row should be equal to numbers of column names in the first row )and place them in subsequent rows as each. in this small example there are 4 column names and so the rest of rows have 4 should have 4 digits output:

snp200 snp1 snp100 snp32
1 1 0 2
0 0 0 2
2 2 2 1 
...

any suggestion by considering that the real data is indeed huge(more than 32000 column names)

  • more than 32000 names, so if the names and data are all on the same line, the number of fields on that line will be even bigger. I wonder if that will start hitting some limit on line lengths for some tools. – ilkkachu Oct 6 '16 at 15:05
3

One option is to use a perl regex, like this:

cat in.txt | perl -pe 's/(([^ ]+ +){4})/$1\n/g' > out.txt

The regular expression said to find one or more not spaces followed by one or more spaces and group the previous 2 things in a set of 4 then add a new line after each match.

Test case:

echo "snp200 snp1 snp100 snp32 1 13454356 0 2 0 0 0 2 2 2 2 1" | perl -pe 's/(([^ ]+ +){4})/$1\n/g'
snp200 snp1 snp100 snp32 
1 13454356 0 2 
0 0 0 2 
2 2 2 1
  • @zara - if you do not know how much data you have how do you expect a computer to know it? – grochmal Oct 6 '16 at 3:03
2

Using BSD's rs(1):

rs 0 4 <data.txt >out.txt
1

If all that you need is four columns:

$ cat data.file | tr ' ' '\n' | columns -w 10 -c 4

snp200    snp1      snp100    snp32
1         1         0         2
0         0         0         2
2         2         2         1
1

With zsh:

$ print -raC4 snp200 snp1 snp100 snp32 1 1 0 2 0 0 0 2 2 2 2 1...
snp200  snp1    snp100  snp32
1       1       0       2
0       0       0       2
2       2       2       1...

POSIXly:

$ printf '%-7s %-7s %-7s %-7s\n' snp200 snp1 snp100 snp32 1 1 0 2 0 0 0 2 2 2 2 1...
snp200  snp1    snp100  snp32
1       1       0       2
0       0       0       2
2       2       2       1...

Or if the data is in a file, with tr+paste:

tr -s ' ' '\n' < file | paste - - - -
1

If the number of named labels can vary, we'd better count them first. In Perl, something like this:

perl -lane '
   $n++ while $F[$n] =~ /^[a-z]/; 
   do { 
     print join(" ", @F[$_*$n .. $_*$n + $n - 1])
   } while (++$_*$n <= $#F)' < input

First, increase $n for every field that starts with a letter, giving the number of named fields, then repeatedly print, joined by spaces, fields 0*$n to 1*$n - 1, then 1*$n to 2*$n - 1 etc. as long as $n times the counter is less than the total number of fields on the line. This assumes all the data is on a single line, subsequent lines will be taken as separate data sets.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.