Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a data file, which can have N rows, and each row is composed M elements separated by white space. Currently, I want to separate each row into several segments. In other words, assume the number of segments is 3; then the original file will be separated into 3 files, each of which has N rows and each row has M/3 elements. Besides writing C++ or Java program, Is there any efficient approach that can fulfill this task on Unix/Linux?

share|improve this question
add comment

4 Answers 4

up vote 1 down vote accepted

This caters for a varying number of fields in the same file, and the last segment being only partialy filled, ie less fields than specified (per segment).
Note though, that if the number of fields in a line results in fewer segments than specified, nothing is written to the output file for those shortfall segments.

awk -v 'ncol=5' -v 'pfix=file' '{
    fldn = 0
    sfix = 1
    segs = NF/ncol
    # round up if number of field is not evenly divisible by number of columns    
    segs = (segs == int(segs)) ?segs :int(segs)+1   
    while (fldn != NF) {
        fmod = (++fldn) % ncol
        printf "%s%s", dlim, $(fldn) >> pfix sfix 
        if (fmod == 1 ) { dlim = " " }
        if ((fmod==0 ) || (fldn==NF))  { 
            printf "\n" >> pfix sfix 
            dlim = ""; sfix++ 
        }
    } 
}' infile
share|improve this answer
add comment

sep_file.ksh

#!/bin/ksh

FILENAME=$1
SEG=$2

SEG_NO=1

while [[ $SEG_NO -le $SEG ]]
do
  awk '{CL=NF/'"$SEG"';CL=(CL==int(CL)?CL:int(CL)+1);LS=(('"$SEG_NO"'-1)*CL)+1;LE=LS+CL-1;if(LE>NF)LE=NF;for(i=LS;i<=LE;i++)printf("%s ",$i);printf("\n")}' $FILENAME > ${FILENAME}_$SEG_NO
  SEG_NO=`echo "$SEG_NO + 1"|bc`
done

Usage: ./sep_file.ksh <file_name_to_read> <no_of_segments>

share|improve this answer
add comment

Is this what you are looking for?

awk '{ print $1 $2 $3 > file1; print $4 $5 $6 > file2; print $7 $8 $9 > file3 }' originalfile

Or did you want something more general?

awk -v 'n=3' -v 'prefix=pref' '{
    for (i = 0; i < n; i++) {
        for (j = 0; j < NF / n; j++) {
            printf("%s ", $(i + j + 1)) > prefix i
        }
        printf("\n") > prefix i
    }
}' originalfile

Note: this relies on the assumption that all rows have the same number of columns.

share|improve this answer
add comment

If your file is clean, I would advice to use the standard application cut

cut has three flags that you should know at least

  • -d to define the delimiter (TAB is the default
  • -f to select the field
  • -c to select a character range

You can choose to either use the combination -d -f, or -c If your file is not TAB delimited, but nicely delimited by space, you can do

cut -d' ' -f1-3

to select the first three columns.

If you want to select the 4th column, that is in between character 25 and 36, you can do

cut -c25-36
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.