How to split a file into multiple file after N appearence of a pattern?

Question

I have a file on Linux, containing the coordinates of thousands of molecules. Each molecule starts with a line containing always the same pattern:

@<TRIPOS>MOLECULE

And then continues with other lines. I would like to split the file into multiple files, each containing a certain number of molecules. What is the easiest way to do this?

Answer 1

One way is to use awk:

awk -v moleculesNum=7 '
/^@<TRIPOS>MOLECULE/{
    if((++num)%moleculesNum==1){
        close(outfile); outfile="file" (++Output)
    }
}
{ print >outfile }' infile

this splits the original file into multiple files each having maximum 7 number of MOLECULEs (adjustable in moleculesNum=7 parameter)

Answer 2

The following is a bash-based csplit utility approach:

### user customization section
tmpdir=$(mktemp -d)
prefix='outfile'
bunch=5
pat='@<TRIPOS>MOLECULE'

## break up the input file on pattern
csplit ./file \
  --silent \
  --elide-empty-files \
  --prefix "$tmpdir/$prefix" \
  --suffix-format='%d.tmp' \
  "/$pat/+1" '{*}' \
;

## coalesce the split up files into bunches
i=0
while :; do
  start=$(( bunch * i ))
  stop=$(( start + bunch - 1 ))
  for ((j=start; j<=stop; j++)) {
    printf '%s\n' "$tmpdir/$prefix$j.tmp"
  } | xargs cat > "./$prefix.$i" 2>/dev/null || break
  (( i++ ))
done

The current directory will hold the outfiles.* bunches.