Say, I have a command command which prints huge number of lines to stdout:


I want to save the output to disk, but not as a single file, but as a sequence of files each having 1000 lines of stdout:




I've tried to google the answer, but every time google points me to tee command, which is useless in this situation. Probably, I'm entering wrong queries.

2 Answers 2


Once you are done saving the file, you could always split the file into file pieces or multiple files based on the number of lines.

split -l 1000 output_file

or even better just try

command | split -l 1000 -

This will split the output stream into files with each 1000 lines (default is 1000 lines without -l option).

The below command will give you additional flexibility to put or enforce a prefix to the filename that will be generated when the output is generated and splitted to store into the file.

command | split -l 1000 - small-

  • I got confused, so for others, its split [arguments...] [input e.g. "-" for stdin] [output_prefix], for example: tar -c somedir | split --byes 100MB --numeric-suffixes --suffix-length=3 - somedir.tar.part- would output a bunch of 100MB files named somedir.tar.part-000, 001, 002 ans so on. Aug 17, 2017 at 19:47
  • @ThorSummoner I suppose it should be --bytes?
    – Qin Heyang
    Nov 5, 2020 at 2:05

You can use a bash script lines.bash

while IFS='' read -r line
  printf -v filename "%04d.txt" "$((a++/1000))"
  echo "$line" >> $filename

and use it as:

cat long_file.txt | bash lines.bash

The only problem I noticed is with * sign in long_file.txt (somebody could correct it).

  • 2
    Set the IFS to empty string to avoid word splitting on read. Use -r to disable backslash escaping on read. Remove -e to avoid backslash escaping on echo. Use quoting to avoid word splitting on echo. Use -v in bash since 4.0 to avoid starting a sub-process. Use post-incrementing as your current code will put in the first file only 999 lines. a=0; while IFS='' read -r line; do printf -v filename "%04d.txt" $((a++/1000)); echo "$line" >> "$filename"; done
    – manatwork
    Dec 6, 2011 at 14:09
  • @manatwork Thank you. Only my printf does not have -v switch. (bash 4.2.10). At least it's not in manpage of printf
    – xralf
    Dec 6, 2011 at 14:21
  • 1
    man printf documents /usr/bin/printf, that could never in life set an environment variable. See help printf for the printf shell built-in's documentation.
    – manatwork
    Dec 6, 2011 at 14:44
  • @manatwork OK. There seems to be syntax error in the ++/ part yet.
    – xralf
    Dec 6, 2011 at 14:48
  • 1
    One more thing: there is no need to use sigil inside arithmetic evaluation, unless you need parameter expansion explicitly. In arithmetic expansion the variables are evaluated anyway.
    – manatwork
    Dec 6, 2011 at 14:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .