0

I am trying to verify if I split large file correctly into multiple pieces. As an toy example, I have a file out_sample:

123456789012
123456789012
123456789012
123456789012
123456789012

This has 64 bytes when stored. When I split this file into six pieces using cut, the resulting files have 15 bytes each:

#!/bin/sh
FILENAME="out_sample"
cut -c1-2 $FILENAME > a
cut -c3-4 $FILENAME > b
cut -c5-6 $FILENAME > c
cut -c7-8 $FILENAME > d
cut -c9-10 $FILENAME > e
cut -c11-12 $FILENAME > f

In total it means 6*15=90 bytes. What is the difference of 26 bytes between size of the original file and sum of sizes of the new files? I noticed that cut appends newline at the end of file, this would probably be 1 byte per file? What about the rest?

  • 1
    Your first file has 13 chars on each line (that is 12 chars + newline) so 5*13=65. Your pieces have 3 chars on each line (that is 2 chars + newline) so 3*5=15. The difference is that you have 6 pieces so you have an additional number of 5*5 newlines (that is 25, which added to 65 equals 90) – don_crissti Sep 8 '15 at 21:57
1

When you cut 2 columns out of a file, cut adds a newline on each line. Therefore each output file consist of 5 lines, with 2 numbers and a newline each.
That explains 25 extra bytes (file one to five; the sixth file got just the newlines added you throw away by cutting column 13 out (newline in the input file)).
There is the last byte comming from? I assume you used an editor that does not force a newline at the very last line of your input file.

So nothing is missing.

1

For this application you probably want to use split rather than cut. split -b 15 $FILENAME should work. It does the bookkeeping automatically, rather than making you calculate file offsets yourself.

EDIT: Misread the question; the split command doesn't split columnwise, just as chunks out of the byte stream. If you really want columnwise, the OP's approach is probably the best available.

  • Huh?  How can you get split to split a file column-wise?  The command you gave doesn't do anything like the commands that the question is about. – Scott Sep 8 '15 at 22:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.