3

How to write a sed (or awk, or both) which will rewrite the following:

echo 'v100 v201 v102 v300 v301 v500 v999 v301' | sed/awk ...

to this output:

v1 v2 v3 v4 v5 v6 v7 v5

i.e. each subsequent vx was rewritten to start with v1...vn and where the same v was used in the sequence (i.e. v301) the same v should be applied (as in v5).

Sidenote: the example input sequence shows all possible eventualities (i.e. duplicates, out of order originals, jumps in original numbers).

Are you the sed or awk expert who can answer this?

5
  • 2
    (A) Why only awk / sed ? (B) Why not Perl / Python ? (C) Does the Input contain only vnnn sequences with spaces , with no other ASCII Characters ? (D) In case the sequence starts with v100 (which gets converted to v1) and has v101, v102 to v199, v200, then that v199 will get converted to v100 in the Output : Is that expected, even though the v100 in the Output is not the same as the v100 in the Input ?
    – Prem
    May 13 at 4:49
  • @Prem (A) Trying to keep it clean Bash only, (B) not external language. (C) no there will be other strings in between. Good point. (D) Yes that is fine. Good point, but in this case in does matter. Thanks May 13 at 11:28
  • 1
    awk or sed have not any more to do with bash than perl. perl made sed and awk obsolete in the mid 80s. May 13 at 11:43
  • 2
    To complement my previous point. awk and sed are as much external languages to that of bash than perl. May 13 at 13:30
  • I can see your point on perl but not on python. Still, sed and awk feel much more bash than perl does. Yes, that's subjective :) May 14 at 0:19

5 Answers 5

5

Using awk:

awk '{ for (i=1; i<=NF; ++i) $i = (seen[$i] ? seen[$i] : seen[$i] = "v" ++n) }; 1'

This goes through all the fields of each input line and reassigns it. The value that it is reassigned is v followed by the next value of the counter n, unless the field's value has been seen before, in which case its new value will be the same as that field's value was given previously.

The 1 at the end triggers the outputting of the modified line.

Testing:

$ echo 'v100 v201 v102 v300 v301 v500 v999 v301' | awk '{ for (i=1; i<=NF; ++i) $i = (seen[$i] ? seen[$i] : seen[$i] = "v" ++n) }; 1'
v1 v2 v3 v4 v5 v6 v7 v5

Alternative awk command that only modifies the field if it matches the regular expression ^v[0-9]+$:

awk '{ for (i=1; i<=NF; ++i) if ($i ~ "^v[0-9]+$") $i = (seen[$i] ? seen[$i] : seen[$i] = "v" ++n) }; 1'

Or, formatted across multiple lines for readability:

awk '
{
    for (i=1; i<=NF; ++i)
        if ($i ~ "^v[0-9]+$")
            $i = (seen[$i] ? seen[$i] : seen[$i] = "v" ++n)
}; 1'
4
  • Wow! Impressive. And matches the original question of only using sed or awk. Solution accepted! May 13 at 11:26
  • Any idea how to deal with in-between text? For example echo 'OTHER v100 WORDS v201 IN BETWEEN v102 THE v300 vVALUES v301 v500 v999 v301'? The in between words will never have the specific format of v followed by numbers with surrounding spaces (or start of line/EOL). May 13 at 11:31
  • 1
    @RoelVandePaar See added bit at the end.
    – Kusalananda
    May 13 at 12:09
  • 1
    I verified the last 'awk' statement against the use case, and it works perfectly, even with varied words and characters in between. Well done and thank you. May 14 at 0:16
5

With perl:

$ echo 'v100 v201 v102 v300 v301 v500 v999 v301' |
   perl -pe 's{v\K\d+}{$seen{$&} //= ++$n}ge'
v1 v2 v3 v4 v5 v6 v7 v5
  • v\d+ matches on v followed by one or more decimal digits. The \K after v resets the start of the matched portion, Keeps what's on its left, the v, so that only the sequence of digits be substituted.
  • the e flag causes the replacement to be treated as code that is evaluated to produce the replacement. In that code, $& contains the matched portion.
  • A // B is a form of OR that expands to A if A is defined and B otherwise (by contrast with A || B which expands to A if A resolves to a true value and B otherwise). //= is the corresponding assignment form. So A //= B is short for if (defined(A)) {A} else {A = B}.

Note that the $seen hash table is indexed on the string values of those numbers, so on v2 v02 v002, you'd get v1 v2 v3 as the 2, 02 and 002 are strings different from each other. You could replace $& with 0+$& to normalise the numbers (010 being treated as 10, not octal 8) so as to get v1 v1 v1 in the example above. Or you could do s{v0*\K\d+}{$seen{$&} //= ++$n}ge to preserve the leading 0s and get v1 v01 v001 instead as a result.

To avoid replacing the v1 found in rev1sion for instance, you can add some word boundary regexp operators on both sides of the match (\bv\K\d+\b). Or to only substitute whitespace-delimited words (to leave v1.2 alone for instance), add some negative look-arounds for non-whiteSpaces: (?<!\S)v\K\d+(?!\S).

0
2

The GNU implementation of awk supports RS being defined as a regular expression and records what it matched in the RT special variable. So with it, you can do something like:

$ echo 'v100 v201 v102 v300 v301 v500 v999 v301' |
   gawk -v RS='v[0-9]+' -v ORS= '
     RT {$0 = $0 "v" (RT in seen ? seen[RT] : seen[RT] = ++n)}1'
v1 v2 v3 v4 v5 v6 v7 v5

Note that it replaces all occurrences of v followed by digits, even those that are found inside a word like in rev1.2 or rev0lution. Like for my perl approach, you may want to adapt it if the numbers may be zero-padded.

1

If your input contains only strings of "v" followed by digits, AND you're OK with space-separated output, this perl script can do what you want:

$ echo 'v100 v201 v102 v300 v301 v500 v999 v301' | 
    perl -lne '
      @line = ();
      #my $i=0;
      #my %seen=();
      while (/(v\d+)/g) {
        $seen{$1} = "v" . ++$i unless ($seen{$1});
        push @line, $seen{$1}
      };
      print join(" ",@line);'
v1 v2 v3 v4 v5 v6 v7 v5

perl's -n option iterates over each line of input (similar to a sed -n script), and -l automatically chomps newlines from the ends of input lines and adds them back to print statements.

The while (/(v\d+)/g) loop iterates over (and captures into $1) all of the strings matching v\d+ in each input line. If that match hasn't been seen before, increment the counter and add it to the %seen hash. Then push (i.e. add it to the end) of an array called @line. After the while loop ends (i.e. after that input line has been processed), print the @line array with a space character between each element.

The @line array is reset to empty for each input line. If you also want the numbering ($i) and the %seen hash to be reset for every input line, uncomment the two lines before the while(...) line:

my $i=0;
my %seen=();
0

GNU awk only:

echo 'v100 v201 v102 v300 v301 v500 v999 v301' |
  awk -v RS='[[:space:]]' -F '' '
    $0 {printf "%s", $1 (A[$0]?A[$0]:A[$0]=++i) RT}'
v1 v2 v3 v4 v5 v6 v7 v5
2
  • 1
    Note that it doesn't work properly if the input start with whitespace. Beware the OP also indicated (though in comments) that the input may contain other things than those vX words. May 13 at 15:35
  • @StéphaneChazelas, thanks, updated
    – nezabudka
    May 13 at 15:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.