Splitting tab-separated file depending on column value

Question

I have a large file that I am trying to split into columns based on the values of the second column. I would like a series of output files that contains the data at the 'switching point'. The switching point is irregular, so I can't split by row number. The data below gives a small extract of the ~0.5M rows

...
605.144302472158    -5.18798828125E-07  0.98388671875
606.144302469633    -5.18798828125E-07  0.98419189453125
606.644327468370    -0.000150299072265625   0.93475341796875
... (~900-1800 rows)
1505.64432519861    -0.000150115966796875   0.4075927734375
1506.14432519735    -0.000150177001953125   0.40753173828125
1507.14435219483    -3.60107421875006E-07   0.41827392578125
1508.14435219230    -3.47900390625003E-07   0.420135498046875
... 

and the output should look like this: file 1

...
    605.144302472158    -5.18798828125E-07  0.98388671875
    606.144302469633    -5.18798828125E-07  0.98419189453125

file 2

606.644327468370    -0.000150299072265625   0.93475341796875
... (900-1800 rows)
1505.64432519861    -0.000150115966796875   0.4075927734375
1506.14432519735    -0.000150177001953125   0.40753173828125

file 3

1507.14435219483    -3.60107421875006E-07   0.41827392578125
1508.14435219230    -3.47900390625003E-07   0.420135498046875
... 

using awk '$2>-0.000001 {print}' input.txt

and awk '$2<-0.000001 {print}' input.txt

I can select the right data, but I don't know how to generate a new file each time $2>-0.000001 switches from 'true' to 'false'.

Using awk '{print > (++a[$2>-0.000001] ".txt")}' input.txt gets me a step closer by printing the output to files, but prints individul files for each row where the condition is met.

I guess I need some sort of while loop, but I'm not sure what it's supposed to look like.

Answer 1

To switch the output to a new file every time a certain criterium is met, you can use a variable storing the name of the output file, and change it every time you see a match for that datum:

$ cat input
17
1
4
13
9
10
7
5
18
19
3
6
16
8
12
15
14
2
11
20
$ awk 'BEGIN { outfilenum=1; outfile="file"outfilenum} $1 > 10 { outfilenum++; outfile = "file"outfilenum} {print >> outfile}' input
$ paste file*
11  20  17  13  18  19  16  12  15  14
        1   9       3   8           2
        4   10      6
            7
            5

To make the matched line appear at the end of each file, simply move the check to after the print routine, rather than before it as shown here.