2

I've read in GNU's docs on "Shell Parameter Expansion" that, given the following syntax:

${parameter//pattern/string}

The pattern is expanded to produce a pattern just as in filename expansion. Parameter is expanded and the longest match of pattern against its value is replaced with string... If there are two slashes separating parameter and pattern..., all matches of pattern are replaced with string.

Given the phrase "all matches of pattern are replaced with string" above, I would expect multiple instances of pattern to be replaced with string all at once, in a single operation, similar to how the /g flag works with a global search-and-replace command in a regex.

I have a piece of open-source code (a function named remove_from_path) which is implemented as follows:

remove_from_path() {
  local path_to_remove="$1"
  local path_before
  local result=":${PATH//\~/$HOME}:"
  local counter=0
  while [ "$path_before" != "$result" ]; do
    counter+=1
    echo "counter: $counter"
    path_before="$result"
    result="${result//:$path_to_remove:/:}"
  done
  result="${result%:}"
  echo "${result#:}"
}

The original code did not include the counter variable- I added that to check how many iterations the while loop would execute.

As we can see, the line result="${result//:$path_to_remove:/:}" uses the same double-forward-slash syntax mentioned in the GNU docs. Given this, I would expect that the while loop would only execute once, since all instances of path_to_remove should be removed from result in one go.

However, that doesn't appear to be the case. In a bash shell (version 3.2.57), I update my $PATH to be the following:

bash-3.2$ PATH="/foo/bar/baz:/foo/bar/baz:/foo/bar/baz:buzz"

I then copy/paste the above function into my shell, and run it. I see the following:

bash-3.2$ remove_from_path "/foo/bar/baz"
counter: 01
counter: 011
counter: 0111
buzz

Please ignore the fact that the incrementing of the counter didn't work the way I expected, since we can still see 3 executions of the while loop. If the ${parameter//pattern/string} double-forward-slash syntax really did replace all matches of pattern with string, why wasn't this accomplished in one iteration of the while loop? Why did we instead need 3 iterations?

0

3 Answers 3

1

The ${var//pattern/replacement} operator from ksh93 (also supported by zsh, bash and mksh) only replaces non-overlapping occurrences of the pattern.

${var//xxx/y} turns xxxxxx into yy, if it turned it into yyyy to replace the 4 overlapping occurrences of xxx in there, that would be quite confusing.

Here $PATH represents a list of directories (and ~ in that regard is the ~ subdirectory of the current working directory, changing it to $HOME is wrong)

Many shells (csh, tcsh, zsh, fish, yash) map it to one of their array variables.

For instance, in zsh, removing all occurrences of a directory from $PATH (mapped to the $path array like in csh) is just a matter of doing:

path=( ${path:#$dir} )

(or path=( "${path[@]:#$dir}" ) to preserve empty elements, but you don't want empty elements in $PATH).

bash doesn't do that, but you can turn $PATH into an array there using the split+glob operator with:

set -o noglob
IFS=:
path=( $PATH'' )

In bash, like in ksh93 or zsh, ${var//pattern/replacement} can be applied to all the elements of an array with the "${array[@]//pattern/replacement}" syntax, but that won't help as that cannot remove elements, just modify them.

So instead, in bash, you're left with looping over the elements:

remove_from_PATH() {
  local - IFS=: dir to_remove result
  set -o noglob
  for dir in $PATH''; do
    for to_remove do
      if [[ $dir = "$to_remove" ]]; then
        continue 2
      fi
    done
    result+=( "$dir" )
  done
  PATH="${result[*]}"
}

(local -, copied from the Almquist shell to make changes to the options (like that set -o noglob) local to the function needs a relatively recent version of bash, won't work with the ancient 3.2 version you seem to be using).


To remove the elements by modifying the :-separated list as stored in $PATH, you'd need for each $to_remove:

  • replace a $to_remove: found at the start with the empty string.
  • replace all :$to_remove: in the middle (some possibly overlapping on the :s) with :
  • remove :$to_remove at the end with the empty string
  • if $PATH only contains $to_remove, then you're out of option, because an empty $PATH means commands are searched in the current directory which is the last thing you want. That should better be handled as an error, or could be just ignored as a pathological case that would not normally happen in real life (like above). Or you can make it /dev/null to ensure $PATH lookup doesn't find anything.

So:

remove_from_PATH() {
  local to_remove dir newpath="$PATH" prev_newpath
  for to_remove do
    while
      prev_newpath=$newpath
      newpath=${newpath#"$to_remove:"}
      newpath=${newpath%":$to_remove"}
      newpath=${newpath//":$to_remove:"/:}
      [[ $newpath != "$prev_newpath" ]]
    do
      continue
    done
  done
  if [[ -n $newpath ]]; then
    PATH=$newpath
  else
    echo >&2 'Refusing to make $PATH empty'
    return 1
  fi
}
1

I think I've found the problem but If I'm wrong please let me now.

In the $result variable you have this string:

:/foo/bar/baz:/foo/bar/baz:/foo/bar/baz:buzz:

And when you apply result="${result//:$path_to_remove:/:}" you are replacing all occurrences of :/foo/bar/baz: with :. But given that pattern the second path is not really matched because of the :. For example:

:/foo/bar/baz:/foo/bar/baz:/foo/bar/baz:buzz:

The bold paths are the occurrences of your pattern.

You can try testing this by using this:

result=':/foo/bar/baz1:/foo/bar/baz2:/foo/bar/baz3:buzz:'
echo "${result//:'/foo/bar/baz'?:/:}"
#Output:
:/foo/bar/baz2:buzz:

As you can see above the second path (/for/bar/baz2) was not affected with the pattern you are using.

So what you can do with the parameter expansion is something like this:

echo "${r//'/foo/bar/baz':/}" # The firsy ':' in the pattern was removed
#and instead of replace the pattern with ':' I'm replacing with nothing.

So your remove_from_path function should look like this:

remove_from_path() {
  local path_to_remove="$1"
  local path_before
  local result=":${PATH//\~/$HOME}:"
  local counter=0
  while [ "$path_before" != "$result" ]; do
    counter+=1
    echo "counter: $counter"
    path_before="$result"
    result="${result//$path_to_remove:/}"
  done
  result="${result%:}"
  echo "${result#:}"
}

However with the logic you have in the function, the loop while will be executed twice. This because the variable path_before is set before the result variable is set with another value by using the parameter expansion.

2
  • 2
    Right, replacement operations like this usually don't look back into the part they already touched, that's also how e.g. s/// in sed works. Anyway, ${r//:'/foo/bar/baz'/} would also remove the leading part of /foo/bar/baz/whatever.
    – ilkkachu
    Oct 29, 2022 at 19:21
  • @ilkkachu Thanks for the clarification and correction! I really was thinking ${result//:$path_to_remove/} would have problems with something but I was not sure. Oct 29, 2022 at 19:35
0

You've got a leading colon too many. Try without:

result="/foo/bar/baz:/foo/bar/baz:/foo/bar/baz:buzz"
echo ${result//$path_to_remove:/:}
:::buzz

and you see it removed all occurrences in one go, no need for a loop. Be aware that fiddling with the PATH system variable may render your session unusable!

1
  • 2
    But since that's PATH, leaving those extra colons there is not good: the empty elements will mean the current directory. ${result//$path_to_remove:/} might be better, though that, too has the issue that e.g. trying to remove /bar would also remove the trailing part of /foo/bar.
    – ilkkachu
    Oct 29, 2022 at 19:08

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .