Split file into n files using csplit (or similar tool)

Question

I have a huge file with the following pattern:

ABC
line 1
line 2
line 3
ABC
line 1
line 2
ABC
line1
ABC
line 1
line 3

Using csplit tool I'm able to split the file above according to /ABC/ pattern into 4 subfiles:

csplit -z input.txt /ABC/ {*}

I wonder how to manually specify the number of desired output files.

Answer 1

You could use awk - not exactly what you wanted, but might do the trick.

Idea: print n lines into a part-file and then search for next occurence of your pattern before creating a new part-file.

Disadvantages:

• If you have large blocks and just skipped the beginning of such a block, some files might become far bigger than others.
• Original file not removed (i.e. twice the space needed).
• As written, the match line has to be exactly ABC (no tolerance vs. other words on same line - could be adjusted)
• Works by setting number of lines rather than desired number of output files (estimate by line number of input file)

akw-script

BEGIN{
    outfile="part_"++i
    j=0
    }
{ 
    j++
    #block size set to at least 10 lines in this example
    #if threshold line number reached: search for next keyword,
    #then increase part file name counter and reset line threshold counter
    if ( j>=10 && $0 == "ABC" ) { outfile="part_"++i ; j=0 }
    print > outfile
}

Execute via

awk -f script.awk input.txt