Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I found a one-line sed command on this site which removes duplicate entries in $PATH. There is one concept that I can't get my mind around. It uses saved matches in the match section to detect the duplicate, and then replaces the match with \1\2.

In the example below, I can't understand why \1 is not always 1111. The test case obviously shows that the \1 match progresses further down the pattern space, but I can't understand why.

sed script (sed_cmd):

p  #debug
:b
s/[:;]\([^:;]*\)\([:;].*\)[;:]\1/;y\1yx\2x/p
s/[yx]//g   #debug remove the field indicators for the next pass
s/;/:/g     #debug
tb
s/^\([^:]*\)\(:.*\):\1/\1\2FixedFirst/
aDone

Test command + output:

echo "0000:1111:2222:3333:4444:1111:2222:3333:0000" | sed -f sed_cmd
0000:1111:2222:3333:4444:1111:2222:3333:0000
0000;y1111yx:2222:3333:4444x:2222:3333:0000
0000:1111;y2222yx:3333:4444x:3333:0000
0000:1111:2222;y3333yx:4444x:0000
0000:1111:2222:3333:4444FixedFirst
Done
share|improve this question

1 Answer 1

The pattern in the substitute command is: [:;]\([^:;]*\)\([:;].*\)[;:]\1. Notice the \1 on the end. This means whatever text matches the first group, \([^:;]*\) must also occur at the end of the pattern.

Your pattern space is initially 0000:1111:2222:3333:4444:1111:2222:3333:0000. The pattern matches :1111:2222:3333:4444:1111, and 1111 matches the first group and also the \1 at the end.

After the first round of substitutions, the pattern space has been changed to 0000:1111:2222:3333:4444:2222:3333:0000. Now, if the 1111 in the pattern space is used to match \([^:;]*\), there is no 1111 left to match \1. Thus, the pattern fails to match and the regex engine tries something else. In this case, using the next available match for the first group, 2222, works.

share|improve this answer
    
Not sure I agree. I placed the "y" and "x" characters into the replacement pattern so I could see what was actually the \1 & \2 matches. the \1 match is always just the digits (no colon). You can see on the 2nd and 3rd pass, the "y" surrounds a different set...and I don't see what makes the match move down the string if the s command always starts at the befinning of the pattern space. I still don't get it. –  dpinvidic Jul 10 at 18:17
    
First, thanks for your help. The first part of the match pattern is /:([^:;].*). So I feel that this should always match ":1111" on every repeat of the substitute command. I don't see how it matches ":2222" on the second pass. –  dpinvidic Jul 10 at 18:30
    
@dpinvidic :1111 and :2222 are very similar. You should be able to easily see that the pattern matches both strings. Your question should not be "why does this pattern match :2222", because that should be obvious (it's almost the same string as :1111), but rather "why does this pattern NOT match :1111 when :1111 comes before :2222". I explained that in my answer. I have edited my answer a bit. Please reread the entire answer carefully. –  jw013 Jul 10 at 18:39
    
Darn...I knew it was something simple. I just forgot the basic fact that SED started each pattern match attempt with one character. If the match fails, it moves to the next character and once again attempts to match the full expression. I had it in my mind that the very first ":1111" was always a match and became \1. Thank You so much....I feel like a Dumb S..t –  dpinvidic Jul 10 at 18:58

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.