0

I have a huge file with the following pattern:

ABC
line 1
line 2
line 3
ABC
line 1
line 2
ABC
line1
ABC
line 1
line 3

Using csplit tool I'm able to split the file above according to /ABC/ pattern into 4 subfiles:

csplit -z input.txt /ABC/ {*}

I wonder how to manually specify the number of desired output files.

5
  • Do you require that the splitting happens at the same points where csplit would have split, i.e. on the ABC lines? – Kusalananda Dec 17 '19 at 6:52
  • @Kusalananda Yes, file should be split according to the /^ABC/ line. – Andrej Dec 17 '19 at 6:55
  • If your split is determined by the content of the file, then I’m not sure what you’re asking for. Can you elaborate further? – bxm Dec 17 '19 at 6:59
  • @bxm Sorry, maybe my question is not clear. In the current setting, the toy file is splitted into four files using the provided csplit command. Is there a way to manually specify the number of output files, e.g. 2? – Andrej Dec 17 '19 at 7:02
  • @bmx They want the splitting to be done on the ABC lines, but not on every such line. – Kusalananda Dec 17 '19 at 7:08
1

You could use awk - not exactly what you wanted, but might do the trick.

Idea: print n lines into a part-file and then search for next occurence of your pattern before creating a new part-file.

Disadvantages:

  • If you have large blocks and just skipped the beginning of such a block, some files might become far bigger than others.
  • Original file not removed (i.e. twice the space needed).
  • As written, the match line has to be exactly ABC (no tolerance vs. other words on same line - could be adjusted)
  • Works by setting number of lines rather than desired number of output files (estimate by line number of input file)

akw-script

BEGIN{
    outfile="part_"++i
    j=0
    }
{ 
    j++
    #block size set to at least 10 lines in this example
    #if threshold line number reached: search for next keyword,
    #then increase part file name counter and reset line threshold counter
    if ( j>=10 && $0 == "ABC" ) { outfile="part_"++i ; j=0 }
    print > outfile
}

Execute via

awk -f script.awk input.txt

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.