11

Bash 4.2 on CentOS 6.5:

In my ~/.bash_profile I have a bunch of aliases, including:

alias grep='grep -n --color=always'

so that I can get color highlighting and print line numbers automatically when running grep. If I run the following, highlighting works as expected:

$ grep -Re 'regex_here' *.py

However, when I ran this recently:

$ find . -name '*.py' | xargs grep -E 'regex_here'

the results were not highlighted and line numbers weren't printed, forcing me to go back and explicitly add -n --color=always to the grep command.

  • Does xargs not read aliases in the environment?
  • If not, is there a way to make it do that?
  • This Q&A has what you want. – psimon Jul 8 '14 at 16:44
  • @psimon right, that's essentially saying to do what I already did in my workaround - I had to manually expand my alias in the xargs command. What I'm trying to find out is if there's a way that I can directly call my alias from xargs. – MattDMo Jul 8 '14 at 16:51
  • 1
    Have you tried export GREP_OPTIONS='-n --color=always' before your xargs command? – doneal24 Jul 9 '14 at 12:47
  • @DougO'Neal thanks, that worked! I'll add that to my .bash_profile. Feel free to write up an answer... – MattDMo Jul 9 '14 at 14:00
8

An alias is internal to the shell where it is defined. It is not visible to other processes. The same goes for shell functions. xargs is a separate application, which is not a shell, so doesn't have a concept of aliases or functions.

You can make xargs invoke a shell instead of invoking grep directly. However just invoking a shell isn't enough, you have to define the alias in that shell as well. If the alias is defined in your .bashrc, you can source that file; however this may not work your .bashrc performs other tasks that don't make sense in a non-interactive shell.

find . -name '*.py' | xargs bash -c '. ~/.bashrc; grep -E regex_here "$@"' _

Beware of the intricacies of nested quoting when typing the regexp. You can simplify your life by passing the regexp as a parameter to the shell.

find . -name '*.py' | xargs bash -c '. ~/.bashrc; grep -E "$0" "$@"' regex_here

You can perform the alias lookup explicitly. Then xargs will see grep -n --color=always.

find . -name '*.py' | xargs "${BASH_ALIASES[grep]}" regex_here

In zsh:

find . -name '*.py' | xargs $aliases[grep] regex_here

By the way, note that find … | xargs … breaks on filenames containing spaces (among others). You can fix this by changing to null-delimited records:

find . -name '*.py' -print0 | xargs -0 "${BASH_ALIASES[grep]}" regex_here

or by using -exec:

find . -name '*.py' -exec "${BASH_ALIASES[grep]}" regex_here {} +

Instead of calling find, you can do everything entirely inside the shell. The glob pattern **/ traverses directories recursively. In bash, you need to run shopt -s globstar to enable this glob pattern first.

grep regex_here **/*.py

This has a few limitations:

  • If a lot of files match (or if they have long paths), the command may fail because it exceeds the maximum command line length.
  • In bash ≤4.2 (but not in more recent versions, nor in ksh or zsh), **/ recurses into symbolic links to directories.

Another approach is to use process substitution, as suggested by MariusMatutiae.

grep regex_here <(find . -name '*.py')

This is useful when **/ isn't applicable: for complex find expressions, or in bash ≤4.2 when you don't want to recurse under symbolic links. Note that this breaks on file names containing spaces; a workaround is to set IFS and disable globbing, but it's starting to get a bit complex:

(IFS=$'\n'; set -f; grep regex_here <(find . -name '*.py') )
  • thank you for the clear explanation of why aliases aren't visible to other processes – MattDMo Jul 9 '14 at 1:40
  • One may also use process substitution, see my answer. – MariusMatutiae Jul 9 '14 at 15:31
10

Use alias xargs='xargs '

alias: alias [-p] [name[=value] ... ]
(snip)
A trailing space in VALUE causes the next word to be checked for
alias substitution when the alias is expanded.
  • Thanks for that, I didn't know about the trailing space trick. – MattDMo Nov 21 '15 at 1:56
  • Np. It's also useful with sudo – 1.61803 Nov 21 '15 at 2:31
2

Please take this as a demonstration of another approach, which I cannot find in the the related SO question:

You can write a wrapper function for xargs which checks if the first argument is an alias and if so, expand it accordingly.

Here is a code which does exactly that but unfortunately it requires the Z shell and hence does not run 1:1 with bash (and frankly, I'm not used to bash enough to port it) :

xargs () {
        local expandalias
        if [[ $(which $1) =~ "alias" ]]; then
                expandalias=$(builtin alias $1) 
                expandalias="${${(s.'.)expandalias}[2]}"
        else
                expandalias=$1
        fi
        command xargs ${(z)expandalias} "${(z)@[2,-1]}"
}

Proof, that it works:

zsh% alias grep="grep -n"´                          # include line number of match
zsh% find foo -name "*.p*" | xargs grep -E test
foo/bar.p0:151:#data=test
foo/bar.p1:122:#data=test                           # line numbers included
zsh% unalias grep
zsh% find foo -name "*.p*" | xargs grep -E test
foo/bar.p0:#data=test
foo/bar.p1:#data=test                               # line numbers not included
zsh% 
1

A simpler, and more elegant solution, is to use process substitution:

grep -E 'regex_here' <( find . -name '*.py')

It does not create a new shell like the pipe does, which means you are still in your original shell where the alias is defined, and the output is exactly what you wish it to be.

Just be careful to leave no space between redirection and parenthesis, otherwise bash will throw an error. To the best of my knowledge, process substitution is supported by Bash, Zsh, Ksh{88,93}, but not by pdksh (I am told that should be a not yet).

  • I think pdksh development is dead. Mksh is more or less a successor project — “pretty hard, it turns out (parsing concept is done in tg@’s head)”. – Gilles Jul 9 '14 at 17:25
  • Process substitution is a good method for complex find commands, though you need to beware that it'll break on spaces, and that can't be fixed as easily as find | xargs can be (by switching to -print0 and -0, or using -exec). When applicable, **/ is simpler and more robust. – Gilles Jul 9 '14 at 17:27
0

grep will read a set of default options from the environment variable GREP_OPTIONS. If you will put

 export GREP_OPTIONS='--line-number --color=always'

in your .bashrc then the variable will be passed on to subshells and you'll get the results you expect.

  • But do not put --line-number or --color=always in GREP_OPTIONS unless it's just for one command, this will break a lot of scripts. --color=auto is ok to have there, and that's about all. Putting this line in your .bashrc will break a lot of stuff. – Gilles Jul 9 '14 at 17:28
  • @Gilles Setting any aliases or overriding default options for any command for the root account is a bad thing. Setting these options for a user account is unlikely to cause many problems. I cannot come up with any user scripts that are problematic. – doneal24 Jul 9 '14 at 18:55
  • Just about any script that uses grep in a way that goes beyond testing the presence of an occurrence will break. For example, from /etc/init.d/cron on my system: value=`egrep "^${var}=" "$ENV_FILE" | tail -n1 | cut -d= -f2` . Or from /usr/bin/pdfjam: pdftitl=`printf "%s" "$PDFinfo" | grep -e … | sed -e …` . An alias is not a problem since it isn't seen in scripts. – Gilles Jul 9 '14 at 19:44
  • @Gilles I'm aware of many scripts like this. The ones I can't get around are generally run only by root (like /etc/init.d/cron). Personally, I don't have any aliases defined on my user accounts nor do I set options in a rc file or via environment variables to override the default behavior of commands. I prefer predictability over convenience. – doneal24 Jul 9 '14 at 19:46
  • Aliases don't break predictability since they aren't seen by scripts. Setting GREP_OPTIONS breaks predictability very badly, except for a few options like --color=auto (which is what it was designed for). – Gilles Jul 9 '14 at 19:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.