1

Deep within a bash script I have

grep -Ech 'string1|string2' /server/directory/$servername_log_20150312* | awk ...

I was getting the "too many arguments" error so I turned on set -x and I see that the files grep is attempting to search start with files like

/server/directory/$servername_log_20150101.000001.log   # Notice the date disparity.

The $servername in each string is being properly expanded with the value for that iteration of the loop. Why is grep not limiting itself to file names beginning with the string I specified?

  • 1
    You sure the script says log_20150312*? Maybe you made a type and it actually says log_20150312 * – nanny Mar 13 '15 at 18:13
  • I've just confirmed that there is no space. – WAF Mar 13 '15 at 18:14
  • 5
    What is the value of the shell variable $servername_log_20150312? Or, did you mean to use ${servername}_log_20150312? For example, note that, if $servername_log_20150312 is empty, then $servername_log_20150312* will expand to all files in the directory, regardless of name. – John1024 Mar 13 '15 at 18:23
  • There's the bug. I was fooled because it looked like it was recognizing the server names, but it was just grabbing them all as you say. Some "s should fix this! – WAF Mar 13 '15 at 18:26
4

The Issue

Let us start by defining a server name:

$ servername=SomeName

Now try:

$ echo "$servername_log_20150312"

$

The above returns nothing because (a) underline is a legal character in a shell variable name, and (b) we never defined servername_log_20150312.

Now consider:

/server/directory/$servername_log_20150312*

After variable expansion, this becomes:

/server/directory/*

After pathname expansion, the above becomes every file in the directory.

Two solutions

Because this sort of thing is common, the shell has brace notation for variables:

$ echo "${servername}_log_20150312"
SomeName_log_20150312

Other ways of separating the name from the characters which follow are possible:

$ echo "$servername""_log_20150312"
SomeName_log_20150312

Documentation

From man bash, a variable name may contain any combination of alphanumeric characters and underscores but must start with an alphabetic character:

   name   A word consisting only of alphanumeric characters  and  underscores,  and
          beginning  with  an alphabetic character or an underscore.  Also referred
          to as an identifier.

POSIX makes limits on variable names implementation dependent. While it requires that shells accept alphanumerics and underscores in names, it seemingly gives shells the option of accepting anything except = and NUL.

1

@john1024 correctly pointed out the bug in my script. When running the grep command, the shell was looking for the variable $servername_log_20150312 rather than just $servername as intended.

Offsetting it as "$servername" solved the problem.

I thought it was grep acting strangely because I saw all of the expected server names in the input to grep. But that was a trick because it was in fact failing to find the variable $servername_log_20150312 and defaulting to searching the whole directory. As it happened, all of the desired server names showed up in the directory before grep's maximum argument limit was reached, further obscuring the true cause.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.