1

Im trying to split a csv file into multiple files using the below command..This command splits into 5 files but the data is truncated in one of the file?

How do I fix this issue?

>>split -n 5 -d -a 2  testfile

I have 5001 lines in my input but the split splits into

file1 - 932
file2 - 910
file3 - 1149
file4 - 1044
file5 - 966

Shouldn't it be 1000 lines per file?

  • 1
    I hope you realize that the output of a split (with -n but also with -l) operation on a CSV file is not necessarily a valid CSV file. Of course it might not matter to you, or not apply to your particular input. – Anthon Apr 29 '14 at 4:15
  • I don't understand why spliting using '-l' (number of lines) might not be valid CSV? Because the header row is only present in the first file? Or something else? – Jonathan Hartley Nov 27 '14 at 15:48
4

split -n 5 splits the file into five parts making all but the last part have the same number of bytes.

If you want to do splitting only on line boundaries, use:

split -n l/5 -d -a 2  testfile

This is detailed in the GNU docs for split as follows:

‘-n chunks’
‘--number=chunks’
     Split input to chunks output files where chunks may be:

          n      generate n files based on current size of input
          k/n    only output kth of n to stdout
          l/n    generate n files without splitting lines
          l/k/n  likewise but only output kth of n to stdout
          r/n    like ‘l’ but use round robin distribution
          r/k/n  likewise but only output kth of n to stdout

Another option is to specify how many lines you want in each output file:

split -l 1000 -d -a 2  testfile

The above splits the file into parts, each (except the last) with 1,000 lines. This is documented as:

‘-l lines’
‘--lines=lines’
     Put lines lines of input into each output file.

3

Use the -l option of split. From the man page:

 -l line_count
         Create smaller files n lines in length.

e.g.

split -l 1000 afile.txt
  • What is the problem with -n option? – user1050619 Apr 29 '14 at 3:19
  • From the man page -n is for chunks so, it will ignore newlines I suppose. – mkc Apr 29 '14 at 3:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.