0

I have a huge file with the following pattern:

ABC
line 1
line 2
line 3
ABC
line 1
line 2
ABC
line1
ABC
line 1
line 3

Using csplit tool I'm able to split the file above according to /ABC/ pattern into 4 subfiles:

csplit -z input.txt /ABC/ {*}

I wonder how to manually specify the number of desired output files.

5
  • Do you require that the splitting happens at the same points where csplit would have split, i.e. on the ABC lines?
    – Kusalananda
    Commented Dec 17, 2019 at 6:52
  • @Kusalananda Yes, file should be split according to the /^ABC/ line.
    – Andrej
    Commented Dec 17, 2019 at 6:55
  • If your split is determined by the content of the file, then I’m not sure what you’re asking for. Can you elaborate further?
    – bxm
    Commented Dec 17, 2019 at 6:59
  • @bxm Sorry, maybe my question is not clear. In the current setting, the toy file is splitted into four files using the provided csplit command. Is there a way to manually specify the number of output files, e.g. 2?
    – Andrej
    Commented Dec 17, 2019 at 7:02
  • @bmx They want the splitting to be done on the ABC lines, but not on every such line.
    – Kusalananda
    Commented Dec 17, 2019 at 7:08

1 Answer 1

1

You could use awk - not exactly what you wanted, but might do the trick.

Idea: print n lines into a part-file and then search for next occurence of your pattern before creating a new part-file.

Disadvantages:

  • If you have large blocks and just skipped the beginning of such a block, some files might become far bigger than others.
  • Original file not removed (i.e. twice the space needed).
  • As written, the match line has to be exactly ABC (no tolerance vs. other words on same line - could be adjusted)
  • Works by setting number of lines rather than desired number of output files (estimate by line number of input file)

akw-script

BEGIN{
    outfile="part_"++i
    j=0
    }
{ 
    j++
    #block size set to at least 10 lines in this example
    #if threshold line number reached: search for next keyword,
    #then increase part file name counter and reset line threshold counter
    if ( j>=10 && $0 == "ABC" ) { outfile="part_"++i ; j=0 }
    print > outfile
}

Execute via

awk -f script.awk input.txt

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .