Given a sorted input file (or command output) that contains unique numbers, one per line, I would like to collapse all runs of consecutive numbers into ranges such that

n
n+1
...
n+m

becomes

n,n+m

input sample:

2
3
9
10
11
12
24
28
29
33

expected output:

2,3
9,12
24
28,29
33
  • Very nice question, but would benefit from some clarifications for newbies. Please ! – user90704 Sep 19 at 17:46
  • thank you. just not clear to me what it means n then n+1. given first number is 2 then I assume first range 2,3 then second range should be 9,10 then 10-11 how you calculated the "expected output" ranges. Would be appreciated if you explain. – user90704 Sep 19 at 19:44
  • @TNT - consecutive ="following each other continuously" (I'm 100% sure consecutive has the same meaning in all languages) so when I say collapse all consecutive numbers I mean all consecutive numbers like n, n+1, n+2, n+3 and so on (à la n++) up to n+m should be combined into a single range: n,n+m. So combine as many numbers as possible as long as they are consecutive. – don_crissti Sep 19 at 20:21
  • Thank you for explaining what consecutive means and it is, indeed, has the same meaning in all languages. Would you please clarify what combined into means? For instance how you get 9-12, 28-29. I am not a genius mathematician nor a programmer. As a reviewer or reader, the question must be clear for general people. Thank you for any additional clarifications! – user90704 Sep 20 at 9:15

12 Answers 12

awk '
    function output() { print start (prev == start ? "" : ","prev) }
    NR == 1 {start = prev = $1; next}
    $1 > prev+1 {output(); start = $1}
    {prev = $1}
    END {output()}
'

With dc for the mental exercise:

dc -f "$1" -e '
[ q ]sB
z d 0 =B sc sa z sb
[ Sa lb 1 - d sb 0 <Z ]sZ
lZx
[ 1 sk lf 1 =O lk 1 =M ]sS
[ li p c 0 d sk sf ]sO
[ 2 sf lh d sj li 1 + !=O ]sQ
[ li n [,] n lj p c 0 sf ]sM
[ 0 sk lh sj ]sN
[ 1 sk lj lh 1 - =N lk 1 =M ]sR
[ 1 sf lh si ]sP
[ La sh lc 1 - sc lf 2 =R lf 1 =Q lf 0 =P lc 0 !=A ]sA
lAx
lSx
'
  • 2
    👍 this is the kind of answer that makes you wish you could upvote twice... – don_crissti Sep 21 at 19:18
  • 1
    @don_crissti, if you're in to this sort of stuff, post the same question on codegolf.se and someone will implement it in Brainf**k. – ilkkachu Sep 24 at 21:12
  • lol this ****head (whoever he is) must hate me really bad if he downvoted your answer just because I said I liked it... Some pretty sick minds around here, I swear... – don_crissti Oct 1 at 17:02

awk, with a different (more C-like) approach:

awk '{ do{ for(s=e=$1; (r=getline)>0 && $1<=e+1; e=$1); print s==e ? s : s","e }while(r>0) }' file

the same thing, even less awk-ward:

awk 'BEGIN{
    for(r=getline; r>0;){
        for(s=e=$1; (r=getline)>0 && $1<=e+1; e=$1);
        print s==e ? s : s","e
    }
    exit -r
}' file
  • Nice. In the interests of compactness, for(r=getline; r>0;) could just be for(r=getline;r;) – steve Sep 21 at 18:56
  • And (r=getline)>0 could just be (r=getline) – steve Sep 21 at 18:56
  • 1
    @steve getline returns -1 on error (eg EIO) – mosvy Sep 21 at 19:02
  • 1
    @steve. That's why the exit -r too -- that could be removed (awk will handle that itself on the next automatic getline) but I wanted the second version to be completely unmagical. – mosvy Sep 21 at 19:14
  • on my gnu awk, getline returns zero on EIO. Example : echo foo | awk 'BEGIN{a=getline;print a;a=getline;print a}' yields output of "1" followed by "0". Man page : "The getline command returns 1 on success, 0 on end of file, and -1 on an error" – steve Sep 22 at 8:37

Using Perl substitute with eval (Sorry for the obfuscation...):

perl -0pe 's/(\d+)\n(?=(\d+))/ $1+1==$2 ? "$1," : $& /ge; 
           s/,.*,/,/g' ex
  • first substitution creates lines with "," separated consecutive int sequences;
  • second substitution, removes middle numbers.

Another awk approach (a variation of glenn's answer):

awk '
    function output() { print start (start != end? ","end : "") }
    end==$0-1 || end==$0 { end=$0; next }
    end!=""{ output() }
    { start=end=$0 }
END{ output() }' infile

An alternative in awk:

<infile sort -nu | awk '
     { l=p=$1 }
     { while ( (r=getline) >= 0 ){
           if ( $1 == p+1 ) { p=$1;  continue };
           print ( l==p ? l : l","p );
           l=p=$1
           if(r==0){ break };
           }
       if (r == -1 ) { print "Unexpected error in reading file"; quit }
     }
    ' 

On one line (no error check):

<infile awk '{l=p=$1}{while((r=getline)>=0){if($1==p+1){p=$1;continue};print(l==p?l:l","p);l=p=$1;if(r==0){ break };}}'

With comments (and pre-processing the file to ensure a sorted, unique list):

<infile sort -nu | awk '

     { l=p=$1 }    ## Only on the first line. The loop will read all lines.

     ## read all lines while there is no error.
     { while ( (r=getline) >= 0 ){

           ## If present line ($1) follows previous line (p), continue.
           if ( $1 == p+1 ) { p=$1;  continue };

           ### Starting a new range ($1>p+1): print the previous range.
           print ( l==p ? l : l","p );

           ## Save values in the variables left (l) and previous (p).
           l=p=$1

           ## At the end of the file, break the loop.
           if(r==0){ break };

           }

       ## All lines have been processed or got an error.
          if (r == -1 ) { print "Unexpected error in reading file"; quit }
     }
    ' 

Yet another awk solution similar to the other:

#!/usr/bin/awk -f

function output() {
    # This function is called when a completed range needs to be
    # outputted. It will use the global variables rstart and rend.

    if (rend != "")
        print rstart, rend
    else
        print rstart
}

# Output field separator is a comma.
BEGIN { OFS = "," }

# At the start, just set rstart and prev (the previous line's number) to
# the first number, then continue with the next line.
NR == 1 { rstart = prev = $0; next }

# Calculate the difference between this line and the previous. If it's
# 1, move the end of the current range here.
(diff = $0 - prev) == 1 { rend = $0 }

# If the difference is more than one, then we're onto a new range.
# Output the range that we were processing and reset rstart and rend.
diff > 1 {
    output()

    rstart = $0
    rend = ""
   }

# Remember this line's number as prev before moving on to the next line.
{ prev = $0 }

# At the end, output the last range.
END { output() }

The rend variable is not actually needed, but I wanted to keep as much range logic as possible away from the output() function.

How about

awk '
$0 > LAST+1     {if (NR > 1)  print (PR != LAST)?"," LAST:""
                 printf "%s", $0
                 PR = $0
                }
                {LAST  = $0
                }
END             {print (PR != LAST)?"," LAST:""
                }
' file
2,3
9,12
24
28,29
33
  • I'd stick to lower case variable names, but that's just style. I came up with the same logic. – glenn jackman Sep 19 at 17:42
  • 1
    Actually, there is a bug here: if the first line is less than one, the first condition will be false, so the the first number will not be printed. You need a separate rule for NR==1 – glenn jackman Sep 19 at 17:44
  • I might recheck. I'm not too happy with the clumsy logics anyhow. – RudiC Sep 19 at 18:06

Perl approach!

#!/bin/perl
    print ranges(2,3,9,10,11,12,24,28,29,33), "\n";

sub ranges {
    my @vals = @_;
    my $first = $vals[0];
    my $last;
    my @list;
    for my $i (0 .. (scalar(@vals)-2)) {
        if (($vals[$i+1] - $vals[$i]) != 1) {
            $last = $vals[$i];
            push @list, ($first == $last) ? $first : "$first,$last";
            $first = $vals[$i+1];
        }
    }
    $last = $vals[-1];
    push @list, ($first == $last) ? $first : "$first,$last";
    return join ("\n", @list);
}
  • 2
    24,24 and 33,33 is not quite what was requested... – RudiC Sep 19 at 18:12
  • As your awk approach is quite similar to glenn jackman's and mine, refer to those. – RudiC Sep 19 at 20:11

Ugly software tools bash shell code, where file is the input file:

diff -y file <(seq $(head -1 file) $(tail -1 file))  |  cut -f1  | 
sed -En 'H;${x;s/([0-9]+)\n([0-9]+\n)*([0-9]+)/\1,\3/g;s/\n\n+/\n/g;s/^\n//p}'

Or with wdiff:

wdiff -12 file <(seq $(head -1 file) $(tail -1 file) ) | 
sed -En 'H;${x;s/([0-9]+)\n([0-9]+\n)*([0-9]+)/\1,\3/g;s/=+\n\n//g;s/^\n//p}'

How these work: Make a gapless sequential list with seq using the first and last numbers in the input file, (because file is already sorted), and diff does most of the work. The sed code is mainly just formatting, and replacing in-between numbers with a comma.

For a related problem, which is the inverse of this one, see: Finding gaps in sequential numbers

A nice discussion from 2001 on perlmonks.org, and adapted to read from STDIN or files named on the command line (as Perl is wont to do):

#!/usr/bin/env perl
use strict;
use warnings;
use 5.6.0;  # for (??{ ... })
sub num2range {
  local $_ = join ',' => @_;
  s/(?<!\d)(\d+)(?:,((??{$++1}))(?!\d))+/$1-$+/g;
  tr/-,/,\n/;
  return $_;
}
my @list;
chomp(@list = <>);
my $range = num2range(@list);
print "$range\n";

On a "Unix & Linux" site, a simple, readable, pure (bash) shell script feels most appropriate to me:

#!/bin/bash

inputfile=./input.txt

unset prev begin
while read num ; do
    if [ "$prev" = "$((num-1))" ] ; then
        prev=$num
    else
        if [ "$begin" ] ; then
            [ "$begin" = "$prev" ] && echo "$prev" || echo "$begin,$prev"
        fi
        begin=$num
        prev=$num
    fi
done < $inputfile
  • Sorry, I never upvote answers that use shell loops to process text: it is not only wrong, it's also damn slow to do it with shell loops; take-home message: if you're using a shell loop to process text then you're doing it wrong. – don_crissti Oct 1 at 10:43
  • How come to you 'dc' is the right tool to process text? – Hkoof Oct 1 at 11:46
  • The text in this particular case consists of numbers and the task involves arithmetic which makes dc a suitable tool for the job. – don_crissti Oct 1 at 15:58

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.