Bash: sed or awk rewrite of numerical sequence

Question

How to write a sed (or awk, or both) which will rewrite the following:

echo 'v100 v201 v102 v300 v301 v500 v999 v301' | sed/awk ...

to this output:

v1 v2 v3 v4 v5 v6 v7 v5

i.e. each subsequent vx was rewritten to start with v1...vn and where the same v was used in the sequence (i.e. v301) the same v should be applied (as in v5).

Sidenote: the example input sequence shows all possible eventualities (i.e. duplicates, out of order originals, jumps in original numbers).

Are you the sed or awk expert who can answer this?

Answer 1

Using awk:

awk '{ for (i=1; i<=NF; ++i) $i = (seen[$i] ? seen[$i] : seen[$i] = "v" ++n) }; 1'

This goes through all the fields of each input line and reassigns it. The value that it is reassigned is v followed by the next value of the counter n, unless the field's value has been seen before, in which case its new value will be the same as that field's value was given previously.

The 1 at the end triggers the outputting of the modified line.

Testing:

$ echo 'v100 v201 v102 v300 v301 v500 v999 v301' | awk '{ for (i=1; i<=NF; ++i) $i = (seen[$i] ? seen[$i] : seen[$i] = "v" ++n) }; 1'
v1 v2 v3 v4 v5 v6 v7 v5

Alternative awk command that only modifies the field if it matches the regular expression ^v[0-9]+$:

awk '{ for (i=1; i<=NF; ++i) if ($i ~ "^v[0-9]+$") $i = (seen[$i] ? seen[$i] : seen[$i] = "v" ++n) }; 1'

Or, formatted across multiple lines for readability:

awk '
{
    for (i=1; i<=NF; ++i)
        if ($i ~ "^v[0-9]+$")
            $i = (seen[$i] ? seen[$i] : seen[$i] = "v" ++n)
}; 1'

Answer 2

With perl:

$ echo 'v100 v201 v102 v300 v301 v500 v999 v301' |
   perl -pe 's{v\K\d+}{$seen{$&} //= ++$n}ge'
v1 v2 v3 v4 v5 v6 v7 v5

• v\d+ matches on v followed by one or more decimal digits. The \K after v resets the start of the matched portion, Keeps what's on its left, the v, so that only the sequence of digits be substituted.
• the e flag causes the replacement to be treated as code that is evaluated to produce the replacement. In that code, $& contains the matched portion.
• A // B is a form of OR that expands to A if A is defined and B otherwise (by contrast with A || B which expands to A if A resolves to a true value and B otherwise). //= is the corresponding assignment form. So A //= B is short for if (defined(A)) {A} else {A = B}.

Note that the $seen hash table is indexed on the string values of those numbers, so on v2 v02 v002, you'd get v1 v2 v3 as the 2, 02 and 002 are strings different from each other. You could replace $& with 0+$& to normalise the numbers (010 being treated as 10, not octal 8) so as to get v1 v1 v1 in the example above. Or you could do s{v0*\K\d+}{$seen{$&} //= ++$n}ge to preserve the leading 0s and get v1 v01 v001 instead as a result.

To avoid replacing the v1 found in rev1sion for instance, you can add some word boundary regexp operators on both sides of the match (\bv\K\d+\b). Or to only substitute whitespace-delimited words (to leave v1.2 alone for instance), add some negative look-arounds for non-whiteSpaces: (?<!\S)v\K\d+(?!\S).

Answer 3

The GNU implementation of awk supports RS being defined as a regular expression and records what it matched in the RT special variable. So with it, you can do something like:

$ echo 'v100 v201 v102 v300 v301 v500 v999 v301' |
   gawk -v RS='v[0-9]+' -v ORS= '
     RT {$0 = $0 "v" (RT in seen ? seen[RT] : seen[RT] = ++n)}1'
v1 v2 v3 v4 v5 v6 v7 v5

Note that it replaces all occurrences of v followed by digits, even those that are found inside a word like in rev1.2 or rev0lution. Like for my perl approach, you may want to adapt it if the numbers may be zero-padded.

Answer 4

If your input contains only strings of "v" followed by digits, AND you're OK with space-separated output, this perl script can do what you want:

$ echo 'v100 v201 v102 v300 v301 v500 v999 v301' | 
    perl -lne '
      @line = ();
      #my $i=0;
      #my %seen=();
      while (/(v\d+)/g) {
        $seen{$1} = "v" . ++$i unless ($seen{$1});
        push @line, $seen{$1}
      };
      print join(" ",@line);'
v1 v2 v3 v4 v5 v6 v7 v5

perl's -n option iterates over each line of input (similar to a sed -n script), and -l automatically chomps newlines from the ends of input lines and adds them back to print statements.

The while (/(v\d+)/g) loop iterates over (and captures into $1) all of the strings matching v\d+ in each input line. If that match hasn't been seen before, increment the counter and add it to the %seen hash. Then push (i.e. add it to the end) of an array called @line. After the while loop ends (i.e. after that input line has been processed), print the @line array with a space character between each element.

The @line array is reset to empty for each input line. If you also want the numbering ($i) and the %seen hash to be reset for every input line, uncomment the two lines before the while(...) line:

my $i=0;
my %seen=();

Answer 5

GNU awk only:

echo 'v100 v201 v102 v300 v301 v500 v999 v301' |
  awk -v RS='[[:space:]]' -F '' '
    $0 {printf "%s", $1 (A[$0]?A[$0]:A[$0]=++i) RT}'
v1 v2 v3 v4 v5 v6 v7 v5