# Extracting integer subsequences which begin with a repeated number

I have a file which contains a single column of integers. I want to extract from this file the list of all contiguous subsequences (i.e. subsequences occurring in consecutive order) which begin with the same number twice in a row, and which have a length of 12 integers (including overlapping subsequences).

Additionally, any non-integer lines in the file should be ignored/removed and if any sequence reaches the end of the input before 12 integers are reached then the shortened sequence should still be output.

For example, suppose my input file contains the following data:

``````1
junk
1

1
2
3
4
4
5
6
7
8
9
10
11
12
13
14
15
15
16
``````

Then the solution should produce the following output:

``````1 1 1 2 3 4 4 5 6 7 8 9
1 1 2 3 4 4 5 6 7 8 9 10
4 4 5 6 7 8 9 10 11 12 13 14
15 15 16
``````

Notice that the `junk` line and the empty line are ignored, so that the first three `1` rows are treated as contiguous.

• I wouldn't say that you want to find two consecutive numbers. Two consecutive numbers would like `1,2` or `5,6`. I would say that you want to find the same number occurring twice in two consecutive locations. – igal Nov 26 '17 at 21:04
• what if there would be less than 10 numbers after crucial 2 duplicate digits? Should it print the residue to a file anyway? – RomanPerekhrest Nov 26 '17 at 21:04
• What if there are identical consecutive numbers in those 10 lines as well? – Stéphane Chazelas Nov 26 '17 at 21:36

Here is a Python script that does what you want:

``````#!/usr/bin/env python2
# -*- coding: ascii -*-
"""extract_subsequences.py"""

import sys
import re

# Open the file
with open(sys.argv) as file_handle:

# Read the data from the file
# Remove white-space and ignore non-integers
numbers = [
line.strip()
if re.match("^\d+\$", line)
]

# Set a lower bound so that we can output multiple lists
lower_bound = 0
while lower_bound < len(numbers)-1:

# Find the "start index" where the same number
# occurs twice at consecutive locations
start_index = -1
for i in range(lower_bound, len(numbers)-1):
if numbers[i] == numbers[i+1]:
start_index = i
break

# If a "start index" is found, print out the two rows
# values and the next 10 rows as well
if start_index >= lower_bound:
upper_bound = min(start_index+12, len(numbers))
print(' '.join(numbers[start_index:upper_bound]))

# Update the lower bound
lower_bound = start_index + 1

# If no "start index" is found then we're done
else:
break
``````

Suppose your data is in a file called `data.txt`. Then you could run this script like so:

``````python extract_subsequences.py data.txt
``````

Suppose your input file `data.txt` looks like the following:

``````1
1
1
2
3
4
5
6
7
8
9
10
11
12
``````

Then your output would look like this:

``````1 1 1 2 3 4 5 6 7 8 9 10
1 1 2 3 4 5 6 7 8 9 10 11
``````

To save the output to a file, use output redirection:

``````python extract_subsequences.py data.txt > output.txt
``````

`AWK` approach:

Considering only firstly encountered 2 identical consecutive numbers, i.e. it's suitable for multiple extractions but without considering condition that 2 identical consecutive numbers may go within followed 10-number sequence under one's processed slice.

``````awk 'NR==n && \$1==v{ print v ORS \$1 > "file"++c; tail=n+11; next }
{ v=\$1; n=NR+1 }NR<tail{ print > "file"c }' file
``````
• @asmodeus, copy and paste attentively. Yes, it should be executed as one line – RomanPerekhrest Nov 26 '17 at 22:08

### First variant - O(n)

``````awk '
/^[0-9]+\$/{
arr[cnt++] = \$0;
}

END {
for(i = 1; i < cnt; i++) {
if(arr[i] != arr[i - 1])
continue;

last_element = i + 11;
for(j = i - 1; j < cnt && j < last_element; j++) {
printf arr[j] " ";
}
print "";
}
}' input.txt
``````

### Second variant - O(n * n)

``````awk '
BEGIN {
cnt = 0;
}

/^[0-9]+\$/{
if(prev == \$0) {
arr[cnt] = prev;
cnt_arr[cnt]++;
cnt++;
}

for(i = 0; i < cnt; i++) {
if(cnt_arr[i] < 12) {
arr[i] = arr[i] " " \$0;
cnt_arr[i]++;
}
}

prev = \$0;
}

END {
for(i = 0; i < cnt; i++)
print arr[i];
}' input.txt
``````

Output

``````1 1 1 2 3 4 4 5 6 7 8 9
1 1 2 3 4 4 5 6 7 8 9 10
4 4 5 6 7 8 9 10 11 12 13 14
15 15 16
``````
• @asmodeus If you use `awk -f file` method, you should remove `awk '` from the beginning and `' input.txt` from the ending of the script. And run it by this way: `awk -f script.awk input.txt`. Also, you can just copy all command (`awk` script) to the terminal and press `Enter`. – MiniMax Nov 29 '17 at 0:47
• @asmodeus The third way (which I prefer): putting the full command into the `bash` script (named `my_program.sh`, for example), then making it executable by the `chmod u+x my_program.sh` and running it by the `./my_program.sh`. Don't forget the `bash` shebang - `#!/bin/bash` in the beginning of the `my_program.sh` file. – MiniMax Nov 29 '17 at 0:54
• @asmodeus "The second is just hanging not producing any output." It is when you copy all command into the terminal? I checked them just now, using this way. Both work. – MiniMax Nov 29 '17 at 1:05