# How to collapse consecutive numbers into ranges?

Given a sorted input file (or command output) that contains unique numbers, one per line, I would like to collapse all runs of consecutive numbers into ranges such that

``````n
n+1
...
n+m
``````

becomes

``````n,n+m
``````

input sample:

``````2
3
9
10
11
12
24
28
29
33
``````

expected output:

``````2,3
9,12
24
28,29
33
``````

With `dc` for the mental exercise:

``````dc -f "\$1" -e '
[ q ]sB
z d 0 =B sc sa z sb
[ Sa lb 1 - d sb 0 <Z ]sZ
lZx
[ 1 sk lf 1 =O lk 1 =M ]sS
[ li p c 0 d sk sf ]sO
[ 2 sf lh d sj li 1 + !=O ]sQ
[ li n [,] n lj p c 0 sf ]sM
[ 0 sk lh sj ]sN
[ 1 sk lj lh 1 - =N lk 1 =M ]sR
[ 1 sf lh si ]sP
[ La sh lc 1 - sc lf 2 =R lf 1 =Q lf 0 =P lc 0 !=A ]sA
lAx
lSx
'
``````
• 👍 this is the kind of answer that makes you wish you could upvote twice... Commented Sep 21, 2018 at 19:18
• @don_crissti, if you're in to this sort of stuff, post the same question on codegolf.se and someone will implement it in Brainf**k. Commented Sep 24, 2018 at 21:12
• I like this answer, but have two questions: ~~(1) How I provide `dc` the numbers via `STDIN`? I tried to remove `-f "\$1"` and prepend and `echo "\$numbers"` or append `<<< "\$numbers"`, but it didn’t work.~~ I can use `-f -` and then `dc` reads the `STDIN`. (2) How could I replace the newlines with a custom separator? I have already replaced `[,]` with `[-]`, but I have no idea if I replace the newlines with `, ` (of course, I can do it using `sed`, for example). Commented Jun 29, 2021 at 8:56
• @tukusejssirs Not sure I understand well what you want to know? Some implementation of dc don't get option at all, only files so for the 1 you can try echo '2 3 z p' | dc to print the number of values. first the values and after the code. for the 2, take a look at the man page for the commands n , p and c commands. Be aware that the c command is not always present. If the c command don't exist, you must have a trash stack. Commented Jun 29, 2021 at 16:53
• @ctac_, I have solved the first problem by myself (I needed to use `-f -`). The second problem I wish to solve is to replace the newlines with a custom string, so that the output looks like `2-3, 9-12, 24, 28-29, 33` (I copied the example from the OP). Currenly, I can do it only with an additional command (like `sed`), but not directly with `dc`. Note that I have no idea if it is actually possible with `dc`. ;) Commented Jul 1, 2021 at 16:18
``````awk '
function output() { print start (prev == start ? "" : ","prev) }
NR == 1 {start = prev = \$1; next}
\$1 > prev+1 {output(); start = \$1}
{prev = \$1}
END {output()}
'
``````

`awk`, with a different (more `C`-like) approach:

``````awk '{ do{ for(s=e=\$1; (r=getline)>0 && \$1<=e+1; e=\$1); print s==e ? s : s","e }while(r>0) }' file
``````

the same thing, even less awk-ward:

``````awk 'BEGIN{
for(r=getline; r>0;){
for(s=e=\$1; (r=getline)>0 && \$1<=e+1; e=\$1);
print s==e ? s : s","e
}
exit -r
}' file
``````
• Nice. In the interests of compactness, `for(r=getline; r>0;)` could just be `for(r=getline;r;)` Commented Sep 21, 2018 at 18:56
• And `(r=getline)>0` could just be `(r=getline)` Commented Sep 21, 2018 at 18:56
• @steve `getline` returns -1 on error (eg `EIO`)
– user313992
Commented Sep 21, 2018 at 19:02
• @steve. That's why the `exit -r` too -- that could be removed (awk will handle that itself on the next automatic `getline`) but I wanted the second version to be completely unmagical.
– user313992
Commented Sep 21, 2018 at 19:14
• on my gnu awk, getline returns zero on EIO. Example : `echo foo | awk 'BEGIN{a=getline;print a;a=getline;print a}'` yields output of "1" followed by "0". Man page : "The getline command returns 1 on success, 0 on end of file, and -1 on an error" Commented Sep 22, 2018 at 8:37

Using Perl substitute with eval (Sorry for the obfuscation...):

``````perl -0pe 's/(\d+)\n(?=(\d+))/ \$1+1==\$2 ? "\$1," : \$& /ge;
s/,.*,/,/g' ex
``````
• first substitution creates lines with "," separated consecutive int sequences;
• second substitution, removes middle numbers.

Another `awk` approach (a variation of glenn's answer):

``````awk '
function output() { print start (start != end? ","end : "") }
end==\$0-1 || end==\$0 { end=\$0; next }
end!=""{ output() }
{ start=end=\$0 }
END{ output() }' infile
``````

An alternative in awk:

``````<infile sort -nu | awk '
{ l=p=\$1 }
{ while ( (r=getline) >= 0 ){
if ( \$1 == p+1 ) { p=\$1;  continue };
print ( l==p ? l : l","p );
l=p=\$1
if(r==0){ break };
}
if (r == -1 ) { print "Unexpected error in reading file"; quit }
}
'
``````

On one line (no error check):

``````<infile awk '{l=p=\$1}{while((r=getline)>=0){if(\$1==p+1){p=\$1;continue};print(l==p?l:l","p);l=p=\$1;if(r==0){ break };}}'
``````

With comments (and pre-processing the file to ensure a sorted, unique list):

``````<infile sort -nu | awk '

{ l=p=\$1 }    ## Only on the first line. The loop will read all lines.

## read all lines while there is no error.
{ while ( (r=getline) >= 0 ){

## If present line (\$1) follows previous line (p), continue.
if ( \$1 == p+1 ) { p=\$1;  continue };

### Starting a new range (\$1>p+1): print the previous range.
print ( l==p ? l : l","p );

## Save values in the variables left (l) and previous (p).
l=p=\$1

## At the end of the file, break the loop.
if(r==0){ break };

}

## All lines have been processed or got an error.
if (r == -1 ) { print "Unexpected error in reading file"; quit }
}
'
``````

Yet another `awk` solution similar to the other:

``````#!/usr/bin/awk -f

function output() {
# This function is called when a completed range needs to be
# outputted. It will use the global variables rstart and rend.

if (rend != "")
print rstart, rend
else
print rstart
}

# Output field separator is a comma.
BEGIN { OFS = "," }

# At the start, just set rstart and prev (the previous line's number) to
# the first number, then continue with the next line.
NR == 1 { rstart = prev = \$0; next }

# Calculate the difference between this line and the previous. If it's
# 1, move the end of the current range here.
(diff = \$0 - prev) == 1 { rend = \$0 }

# If the difference is more than one, then we're onto a new range.
# Output the range that we were processing and reset rstart and rend.
diff > 1 {
output()

rstart = \$0
rend = ""
}

# Remember this line's number as prev before moving on to the next line.
{ prev = \$0 }

# At the end, output the last range.
END { output() }
``````

The `rend` variable is not actually needed, but I wanted to keep as much range logic as possible away from the `output()` function.

A nice discussion from 2001 on perlmonks.org, and adapted to read from STDIN or files named on the command line (as Perl is wont to do):

``````#!/usr/bin/env perl
use strict;
use warnings;
use 5.6.0;  # for (??{ ... })
sub num2range {
local \$_ = join ',' => @_;
s/(?<!\d)(\d+)(?:,((??{\$++1}))(?!\d))+/\$1-\$+/g;
tr/-,/,\n/;
return \$_;
}
my @list;
chomp(@list = <>);
my \$range = num2range(@list);
print "\$range\n";
``````

There's a `perl` module called Set::IntSpan which already does this (it was originally written in 1996 to collapse lists of article numbers for .newsrc files, which could be enormous).

There is also a similar module for `python` called intspan, but I haven't used it.

Anyway, with perl and Set::IntSpan (and `tr` to get the input data into comma-separated format, and `tr` again to munge the output), this is trivial.

``````\$ tr \$'\n' ',' < input.txt  |
perl -MSet::IntSpan -lne 'print Set::IntSpan->new(\$_)' |
tr ',-' \$'\n,'
2,3
9,12
24
28,29
33
``````

`Set::IntSpan` is packaged for debian and ubuntu as `libset-intspan-perl`, for fedora as `perl-Set-IntSpan`, and probably for other distros too. Also available on CPAN, of course.

``````awk '
\$0 > LAST+1     {if (NR > 1)  print (PR != LAST)?"," LAST:""
printf "%s", \$0
PR = \$0
}
{LAST  = \$0
}
END             {print (PR != LAST)?"," LAST:""
}
' file
2,3
9,12
24
28,29
33
``````
• I'd stick to lower case variable names, but that's just style. I came up with the same logic. Commented Sep 19, 2018 at 17:42
• Actually, there is a bug here: if the first line is less than one, the first condition will be false, so the the first number will not be printed. You need a separate rule for `NR==1` Commented Sep 19, 2018 at 17:44
• I might recheck. I'm not too happy with the clumsy logics anyhow. Commented Sep 19, 2018 at 18:06

`Perl` approach!

``````#!/bin/perl
print ranges(2,3,9,10,11,12,24,28,29,33), "\n";

sub ranges {
my @vals = @_;
my \$first = \$vals[0];
my \$last;
my @list;
for my \$i (0 .. (scalar(@vals)-2)) {
if ((\$vals[\$i+1] - \$vals[\$i]) != 1) {
\$last = \$vals[\$i];
push @list, (\$first == \$last) ? \$first : "\$first,\$last";
\$first = \$vals[\$i+1];
}
}
\$last = \$vals[-1];
push @list, (\$first == \$last) ? \$first : "\$first,\$last";
return join ("\n", @list);
}
``````
• `24,24` and `33,33` is not quite what was requested... Commented Sep 19, 2018 at 18:12
• As your awk approach is quite similar to glenn jackman's and mine, refer to those. Commented Sep 19, 2018 at 20:11

Ugly software tools `bash` shell code, where file is the input file:

``````diff -y file <(seq \$(head -1 file) \$(tail -1 file))  |  cut -f1  |
sed -En 'H;\${x;s/([0-9]+)\n([0-9]+\n)*([0-9]+)/\1,\3/g;s/\n\n+/\n/g;s/^\n//p}'
``````

Or with `wdiff`:

``````wdiff -12 file <(seq \$(head -1 file) \$(tail -1 file) ) |
sed -En 'H;\${x;s/([0-9]+)\n([0-9]+\n)*([0-9]+)/\1,\3/g;s/=+\n\n//g;s/^\n//p}'
``````

How these work: Make a gapless sequential list with `seq` using the first and last numbers in the input file, (because file is already sorted), and `diff` does most of the work. The `sed` code is mainly just formatting, and replacing in-between numbers with a comma.

For a related problem, which is the inverse of this one, see: Finding gaps in sequential numbers

On a "Unix & Linux" site, a simple, readable, pure (bash) shell script feels most appropriate to me:

``````#!/bin/bash

inputfile=./input.txt

unset prev begin
if [ "\$prev" = "\$((num-1))" ] ; then
prev=\$num
else
if [ "\$begin" ] ; then
[ "\$begin" = "\$prev" ] && echo "\$prev" || echo "\$begin,\$prev"
fi
begin=\$num
prev=\$num
fi
done < \$inputfile
``````
• Sorry, I never upvote answers that use shell loops to process text: it is not only wrong, it's also damn slow to do it with shell loops; take-home message: if you're using a shell loop to process text then you're doing it wrong. Commented Oct 1, 2018 at 10:43
• How come to you 'dc' is the right tool to process text? Commented Oct 1, 2018 at 11:46
• The text in this particular case consists of numbers and the task involves arithmetic which makes `dc` a suitable tool for the job. Commented Oct 1, 2018 at 15:58