0

I have a data file looks like :

1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 3 3 3 3 3  . . .
1 0 4 4 3 1 2 0 0 0 3 1 1 2 1 1 1 1 1 1 0 1 1 3  . . .
0 0 0 0 0 0 0 3 3 1 1 2 3 2 1 2 2 3 1 2 3 1 2 2  . . .          
.
.
.

first I want to insert space among each 5 identical values keeping each 5 identical numbers together in one colum by looking at the first row and then I do not want anz space among those group characters: first step:

1 1 1 1 1  1 1 1 1 1  1  2 2 2 2 2  2 2 2  3 3 3 3 3  . . .
1 0 4 4 3  1 2 0 0 0  3  1 1 2 1 1  1 1 1  1 0 1 1 3  . . .
0 0 0 0 0  0 0 3 3 1  1  2 3 2 1 2  2 3 1  2 3 1 2 2  . . .          
.
.
.

secouns step (output):

11111  11111  1  22222  222  33333  . . .
10443  12000  3  11211  111  10113  . . .
00000  00331  1  23212  231  23122  . . .          
.
.
.

Meanwhile in my real data which is huge I may want to try different group sizes. So I need the script to be flexible .. any suggestion please?

  • 1
    Use sed(1), or a scripting language like Perl or Python. – vonbrand Jan 30 '16 at 17:54
  • 3
    Can you not use the code given to you from your last question? – glenn jackman Jan 30 '16 at 19:44
  • @glennjackman I tried but it did not work. – zara Jan 30 '16 at 19:54
  • 2
    You should show us what you tried so we can help you improve – glenn jackman Jan 30 '16 at 20:03
2

Other variant with awk

awk '
    NR==1{
        for(i=2;i<=NF;i++){
            count++
            if($(i-1)!=$i || count>4){
                D[i]=1
                count=0
            }
        }
     }
     {
        for(i in D)
            $i=" "$i
            print
     }
     ' OFS="" data.file >new.file

And sed

sed -re '
    s/ +//g;s/^/\n/
    ' -f <(
        sed -r '
            s/(. )\1*/s_\\n(&)_\n/g
            s/\S /./g
            s/\n\s*/\\1 \\n_\n/g
            s/\\n[^\n]*\n$/ \\n__/
            1q
        ' data.file
        ) -e '
    s/\S{5}/& /g
    ' data.file >new.file
3

Here's an awk script. You just change the number 5 to have other groupings.

awk '
NR==1{
 previous = $1
 for(i = 1;i<=NF+1;i++)
  if($i!=previous){
    col[++numcol] = i
    previous = $i
  }
}
{ j = 1; start = 1
  for(i = 1;i<NF;i++){
   printf "%s",$i
   if(i==col[j]-1){printf "  "; start = col[j++]}
   else if((i-start+1)%5==0)printf "  "
  }
  printf "%s\n",$NF
}'

The first part just handles line 1, and collects in array col the starting column for each set of same numbers. The second part prints each field with no separation, unless on column 5 from the starting column, or at the end of a sequence.

1

One possible way to do it would be to use perl's unpack function, with a template constructed from the first line of the file.

After removing whitespace, it uses a regular expression with a repeating backreference to find the longest contiguous strings of identical characters not exceeding the given maxwidth, and stores their positions in an array. The fieldwidths are extracted by subtracting adjacent elements of the array, and mapped into a suitable template string of the form A5A5A1A5A3A5A3 to pass to the unpack function.

#!/usr/bin/perl -l

use strict;
use warnings;

my $filename = shift or die "Usage: $0 FILENAME MAXWIDTH\n";
my $maxwidth = shift or die "Usage: $0 FILENAME MAXWIDTH\n";
open my $infile, $filename or die "Could not open $filename: $!";

my $n = $maxwidth-1;
my $template;

while( my $line = <$infile> )  { 
    $line =~ s/\s+//g;

    if ($. == 1) {
        my @ends = (0);
        while ($line =~ /(.)\g1{0,$n}/g) {
            push(@ends, pos $line);
        }
        my @fieldwidths = map $ends[$_] - $ends[$_-1], 1 .. $#ends;
        # http://stackoverflow.com/a/29821158/4440445
        $template =  join "", map { 'A' . $_ } @fieldwidths;
        # http://stackoverflow.com/a/2725663/4440445
    }

    my @fields = unpack($template, $line);
    print join " ", @fields;
}

close $infile;

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.