1

Im trying to split a csv file into multiple files using the below command..This command splits into 5 files but the data is truncated in one of the file?

How do I fix this issue?

>>split -n 5 -d -a 2  testfile

I have 5001 lines in my input but the split splits into

file1 - 932
file2 - 910
file3 - 1149
file4 - 1044
file5 - 966

Shouldn't it be 1000 lines per file?

2
  • 1
    I hope you realize that the output of a split (with -n but also with -l) operation on a CSV file is not necessarily a valid CSV file. Of course it might not matter to you, or not apply to your particular input.
    – Anthon
    Apr 29, 2014 at 4:15
  • I don't understand why spliting using '-l' (number of lines) might not be valid CSV? Because the header row is only present in the first file? Or something else? Nov 27, 2014 at 15:48

2 Answers 2

8

split -n 5 splits the file into five parts making all but the last part have the same number of bytes.

If you want to do splitting only on line boundaries, use:

split -n l/5 -d -a 2  testfile

This is detailed in the GNU docs for split as follows:

‘-n chunks’
‘--number=chunks’
     Split input to chunks output files where chunks may be:

          n      generate n files based on current size of input
          k/n    only output kth of n to stdout
          l/n    generate n files without splitting lines
          l/k/n  likewise but only output kth of n to stdout
          r/n    like ‘l’ but use round robin distribution
          r/k/n  likewise but only output kth of n to stdout

Another option is to specify how many lines you want in each output file:

split -l 1000 -d -a 2  testfile

The above splits the file into parts, each (except the last) with 1,000 lines. This is documented as:

‘-l lines’
‘--lines=lines’
     Put lines lines of input into each output file.

3

Use the -l option of split. From the man page:

 -l line_count
         Create smaller files n lines in length.

e.g.

split -l 1000 afile.txt
2
  • What is the problem with -n option? Apr 29, 2014 at 3:19
  • From the man page -n is for chunks so, it will ignore newlines I suppose.
    – Ketan
    Apr 29, 2014 at 3:21

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .