Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a txt file which is formatted as 250,000 * 3600 (3600 rows and 250,000 column) I am going to split it into 3600 small txt which each file is 250,000 * 1.

I understand the split can basically split large file into small ones with certain size. If I followed the same technique, I would get the same size of files but each file can be either 249,999 * 1 or 250,001 *1.

Can anyone help me on this?

share|improve this question
    
I don't understand what you want to do. Do you want to write each line into a separate file? Does “each file can be either 249,999 * 1 or 250,001 *1” mean that the lines have slightly different lengths, or that you want to cut at places other than line breaks? –  Gilles Nov 6 '12 at 23:13
    
Yes, each line into a separate file. But I want each file is formatted exactly as 250,000 * 1, no more no less. If we only specify the bytes, say 2 kB per file, then we end up with getting 249,999 * 1 and 250,001 * 1 , which I don't want. –  John_Amery Nov 7 '12 at 0:31
    
I still don't get it. Are all your lines the same length or not? If they are, then split -b 250000 does put one line in each file. –  Gilles Nov 7 '12 at 0:39
    
can you tell us how a file with 5 columns * 3 rows is split in the wrong way if you use split? –  miracle173 Nov 7 '12 at 7:55
    
Thanks Gilles. Yes, each line is of the same length. But There are too many lines. It always shows 'too many files' and stops. –  John_Amery Nov 7 '12 at 17:42
add comment

1 Answer

I would suggest writing a small script:

NR_LINES=$(wc -l your-file | awk '{ print $1 }')
for line in {1..$NR_LINES}; do
  head -n $line your-file | tail -n 1 > your-file.part$line
done

This saves the number of lines of your file in the variable NR_LINES. Maybe your file consists of more lines than those you are interested in. In this case you should set the variable for yourself. The script iterates from 1 to the last line in your file. For each run it prints the first n lines ($line). This is piped to tail, which only prints the last line and saves the result in a file.

share|improve this answer
    
Thanks. I would like to cut myfile in every iterate and save every piece. I run this : NR_LINES=$(wc -l data.txt | awk '{ print $1 }') for line in {1..$NR_LINES}; do head -1 $line data.txt | tail -1 1 > data.txt.part$line done –  John_Amery Nov 7 '12 at 0:21
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.