0

I am currently trying to understand parameter expansion, especially the different forms which can remove parts of a parameter's value by pattern matching. For the sake of this question, let's focus on the ${parameter#word} expansion.

The two relevant sections from the manual (man bash) (emphasis mine):

${parameter#word}
${parameter##word}
Remove matching prefix pattern. The word is expanded to produce a pattern just as in pathname expansion, and matched against the expanded value of parameter using the rules described under Pattern Matching below. If the pattern matches the beginning of the value of parameter, then the result of the expansion is the expanded value of parameter with the shortest matching pattern (the '#' case) or the longest matching pattern (the '##' case) deleted. [...]

Pathname Expansion
After word splitting, unless the -f option has been set, bash scans each word for the characters *, ?, and [. If one of these characters appears, then the word is regarded as a pattern, and replaced with an alphabetically sorted list of filenames matching the pattern (see Pattern Matching below). [...]

However, please consider the following part of a terminal session:

root@cerberus ~/scripts # mkdir test
root@cerberus ~/scripts # cd test
root@cerberus ~/scripts/test # touch foo
root@cerberus ~/scripts/test # String=foobar
root@cerberus ~/scripts/test # printf '%s\n' ${String#foo}
bar
root@cerberus ~/scripts/test # printf '%s\n' ${String#*}
foobar
root@cerberus ~/scripts/test #

I am not able to understand the output after the last command. According to the manual, the * should be expanded to foo, because foo is the only file in the current directory; at least, this is my notion of "produce a pattern just as in pathname expansion".

Therefore, in this case, printf '%s\n' ${String#*} should give the same output as printf '%s\n' ${String#foo}.

Obviously, this is not the case. Where is my misunderstanding?

0

2 Answers 2

5

Regarding pattern matching:

* Matches any string, including the null string.

And, in parameter expansion for substring removal, # does the shortest match, while ## the longest.

This is removing the null string (shortest match) from the beginning:

$ printf '%s\n' "${String#*}"
foobar

And this is removing foobar (longest match) from the beginning:

$ printf '%s\n' "${String##*}"

$

Regarding the last paragraph of your description: Substring removal with parameter expansion has nothing to do with what files are in your current directory. The pattern is matching on the parameter value, not any files, think of it as a text processing operation on the parameter's value.

2
  • Thank you very much for the explanation, and +1. Then the wording in the manual is very misleading IMHO ... If I only could give the answer credits twice; @ilkkachu's answer was first.
    – Binarus
    May 10 at 5:46
  • Actually, since the OP (and @ilkkachu, but not you) forgot to quote the parameter expansion, it will be further subject to split+glob in their case, so there will be possible matching against the contents of the file system. For instance, with String='/* /etc/*' and the default value of $IFS, printf '%s\n' ${String#*} would print a list of files in / and /etc May 10 at 7:16
2

It take the "word" part as a pattern like the ones used for filename generation, but it doesn't use it for filename generation. Instead, the pattern there is used to match against the value of the given parameter.

E.g. if you have str=foobsar, then ${str#*o} would try the pattern *o at the start of foobar, find fo as the shortest match (the * matches any number of any character), remove that and expand to obar. Similarly ${str#*o} would find the longest match, foo, remove it at expand bar.

The reason it says "word is expanded", is that you can use expansions in the "word" part, like so:

$ str=foobar
$ char=o
$ echo ${str#*$char}
obar

or

$ echo ${str#$pat}
obar

Then again, with the expansion unquoted as above, the result after the prefix removal would go through word splitting and globbing. You should probably quote the whole expansion to prevent that, i.e. "${str#*o}", or whatever.

6
  • 1
    And filename or pathname expansion, nicknamed globbing, which matches the pattern to files, is done only after word or field splitting which is done only after all parameter substitution (plus tilde expansions, command substitution, arithmetic expansion and process substitution for most bash) is done May 10 at 2:19
  • Thanks for answering that fast, and +1 of course. Then the wording in the manual is very misleading ... I already had wondered anyway what sense my understanding would make at all.
    – Binarus
    May 10 at 5:48
  • @Binarus, yeah, idk, filename generation (or pathname expansion or globbing or whatever we call it) doesn't come to it, so yeah they wouldn't need to reference it. But I suppose Bash's manual doesn't have a more generic name for the patterns, so we get that reference. Or they've been sloppy. (The same pattern match syntax is also used at least in case, and [[ a == b ]], in addition to globbing and ${a#b}/${a%b}/${a/b}, so it is rather general, but maybe they assume people know the pattern match syntax from globbing?)
    – ilkkachu
    May 10 at 7:03
  • 1
    anyway, maybe you'd like the POSIX text more, at least they have a more generic phrase instead of reference globbing, from 2.6.2 Parameter Expansion: "The following four varieties of parameter expansion provide for substring processing. In each case, pattern matching notation (see Pattern Matching Notation), rather than regular expression notation, shall be used to evaluate the patterns." and...
    – ilkkachu
    May 10 at 7:05
  • 1
    ... "${parameter#[word]} Remove Smallest Prefix Pattern. The word shall be expanded to produce a pattern. The parameter expansion shall then result in parameter, with the smallest portion of the prefix matched by the pattern deleted."
    – ilkkachu
    May 10 at 7:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.