60

There is a standard command for file splitting - split.

For example, if I want to split a words file in several chunks of 10000 lines, I can use:

split -dl 10000 words wrd

It would generate several files of the form wrd.01, wrd.02 and so on.

But I want to have a specific extension for those files - for example, I want to get wtd.01.txt, wrd.02.txt files.

Is there a way to do it?

3 Answers 3

88

With gnu split's more recent versions (≥ 8.16), one can use the --additional-suffix switch to have control over the resulting extension. From man split:

--additional-suffix=SUFFIX
              append an additional SUFFIX to file names.

so when using that option:

split -dl 10000 --additional-suffix=.txt words wrd

the resulting pieces will automatically end in .txt:

wrd00.txt
wrd01.txt
.........
7
  • 11
    Not working on mac
    – ericgu
    Feb 10, 2015 at 4:51
  • 2
    I love your sarcasm. I am a unix n00b from the Apple world. I am using OS X Yosemite and I just didn't want others to crash and burn like I did. I tested and reviewed at the docs and we don't have this parameter. I might have missed something. developer.apple.com/library/mac/documentation/Darwin/Reference/…
    – ericgu
    Feb 10, 2015 at 15:46
  • 8
    @swiftshokunin - my answer pertains to gnu split, part of gnu coreutils. It is also available on OSX if you install coreutils via homebrew but note that by default, on OSX, the gnu utilities have a g prepended to their name (e.g. gstat instead of stat) so you invoke it as gsplit (or alter the PATH as per the guide here if you want to use it as split over the OSX split). HTH. Feb 10, 2015 at 16:50
  • 2
    Nice answer. on OS X, use gsplit to get the numeric suffixes (-d) to work. Jul 24, 2015 at 1:02
  • 2
    wow, i had no idea that there is gsplit - it's probably from coreutils mentioned above and it does have --additional-suffix. Thanks to everyone commenting this solution :) May 31, 2016 at 10:15
23

Such tasks are best managed with the shell. Use split and then write a simple loop to rename the files. E.g.

for file in wrd.*
do
    mv "$file" "$file.txt"
done

would rename your wrd.01, wrd.02, etc. files so they all have a .txt extension.

7
  • That's quite obvious, but it would break the conciseness of bash script.
    – Rogach
    Feb 25, 2012 at 5:08
  • 1
    The Unix philosophy is to provide you with a set of simple tools that you then combine to do a job. The "conciseness of the bash script" was not a stated requirement in your question.
    – Kyle Jones
    Feb 25, 2012 at 5:19
  • 7
    PS: the split+mv combo is more that 6 times faster than awk (approx 3s vs 18s) for a 10 million line input file (75 MB)... the text in each line was its own line-number... Thanks for re-stating the "obvious" :)
    – Peter.O
    Feb 25, 2012 at 10:36
  • 3
    PPS: I've just been checking this out a bit further. The speed-difference is related to the number of files created vs the number of formatting and arithmetic calculations awk does for each and every line regardless of the number of output files... Using the same input file as the above example: When there are 100 times fewer files, split + mv is 75 times faster than awk: When there are 100 times more files, split + mv is 1.5 times faster than awk. So, for me, this split + mv method wins, hands down. It is as consice (arguably moreso), and is faster than awk.
    – Peter.O
    Feb 25, 2012 at 15:59
  • 3
    if you're concerned about it being 5 lines long, try this instead: for file in wrd.*; do mv "$file" "$file.txt"; done :)
    – Tony
    Oct 28, 2015 at 4:20
15

Not with split, but you can easily rename them afterwards, or you can do it in awk:

awk '{filename = "wrd." int((NR-1)/10000) ".txt"; print >> filename}' inputfile
2
  • Looks good - but does not work. In your form, complains about "expression for `>>' redirection has null string value", and if "file" is "changed" to "filename", outputs files of the form wrd.{file number}.{line number}.txt (quite a lot of them :)
    – Rogach
    Feb 25, 2012 at 5:15
  • @Rogach Sorry, I hadn't tested it, so I forgot awk doesn't do integer division. I've tested this one.
    – Kevin
    Feb 25, 2012 at 6:07

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .