Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I've been using grep -i more often and I found out that it is slower than its egrep equivalent, where I match against the upper or lower case of each letter:

$ time grep -iq "thats" testfile

real    0m0.041s
user    0m0.038s
sys     0m0.003s
$ time egrep -q "[tT][hH][aA][tT][sS]" testfile

real    0m0.010s
user    0m0.003s
sys     0m0.006s

Does grep -i do additional tests that egrep doesn't?

share|improve this question
12  
Try the grep's the other way around, to make sure you're not measuring the difference between disk caching of the flie. – EightBitTony Mar 14 at 0:26
3  
I have grep'd the file prior to testing, so it is cached. Almost same times if done in reverse order. – tildearrow Mar 14 at 0:28
21  
This can depend on the locale: some locales involve complex calculations to account for case insensitivity. GNU grep is particularly slow in many situations involving Unicode. What locale settings did you use? Under what Unix variant? What is the content of your test file? – Gilles Mar 14 at 0:35
6  
@Gilles looks good, repeating each test here 100 times (timing the entire thing), egrep is faster than grep until I set LANG=C and then they're both roughly the same. – EightBitTony Mar 14 at 0:39
2  
@EightBitTony Look at user time (which does not include time waiting for disk). There is an order of magnitude in difference. – kasperd Mar 15 at 11:17
up vote 68 down vote accepted

grep -i 'a' is equivalent to grep '[Aa]' in an ASCII-only locale. In a Unicode locale, character equivalences and conversions can be complex, so grep may have to do extra work to determine which characters are equivalent. The relevant locale setting is LC_CTYPE, which determines how bytes are interpreted as characters.

In my experience, GNU grep can be slow when invoked in a UTF-8 locale. If you know that you're searching for ASCII characters only, invoking it in an ASCII-only locale may be faster. I expect that

time LC_ALL=C grep -iq "thats" testfile
time LC_ALL=C egrep -q "[tT][hH][aA][tT][sS]" testfile

would produce indistinguishable timings.

That being said, I can't reproduce your finding with GNU grep on Debian jessie (but you didn't specify your test file). If I set an ASCII locale (LC_ALL=C), grep -i is faster. The effects depend on the exact nature of the string, for example a string with repeated characters reduces the performance (which is to be expected).

share|improve this answer
    
The author uses Ubuntu 14.04 which ships with grep 2.10. The speed of case-insensitive matches (-i) with multibyte locales should have improved in 2.17. – Lekensteyn Mar 15 at 12:24
    
@Lekensteyn Good to know, thanks. Ubuntu 14.04 actually comes with grep 2.16, but that's pre-2.17 too; I tested with grep 2.20, which explains why I didn't see the same slowdown. – Gilles Mar 15 at 12:32
    
Right, I was looking at the wrong LTS release, Ubuntu 12.04 ships with grep 2.10 while Ubuntu 14.04 includes grep 2.16. – Lekensteyn Mar 15 at 12:49
1  
I'm quite certain that grep -i 'a' is equivalent to grep '[Aa]' in any locale. The proper example is grep -i 'i' which is either grep '[Ii]'or grep '[İi]' (Uppercase I with dot above, U+130, Turkish locale). However, there is no efficient way for grep to find this equivalence class given a locale. – MSalters Mar 15 at 13:13

Out of curiosity, I tested this on an Arch Linux system:

$ uname -r
4.4.5-1-ARCH
$ df -h .
Filesystem      Size  Used Avail Use% Mounted on
tmpfs           3.9G  720K  3.9G   1% /tmp
$ dd if=/dev/urandom bs=1M count=1K | base64 > foo
$ df -h .                                         
Filesystem      Size  Used Avail Use% Mounted on
tmpfs           3.9G  1.4G  2.6G  35% /tmp
$ for i in {1..100}; do /usr/bin/time -f '%e' -ao grep.log grep -iq foobar foo; done
$ for i in {1..100}; do /usr/bin/time -f '%e' -ao egrep.log egrep -q '[fF][oO][oO][bB][aA][rR]' foo; done

$ grep --version
grep (GNU grep) 2.23
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Mike Haertel and others, see <http://git.sv.gnu.org/cgit/grep.git/tree/AUTHORS>.

And then some stats courtesy of Is there a way to get the min, max, median, and average of a list of numbers in a single command?:

$ R -q -e "x <- read.csv('grep.log', header = F); summary(x); sd(x[ , 1])"
> x <- read.csv('grep.log', header = F); summary(x); sd(x[ , 1])
       V1       
 Min.   :1.330  
 1st Qu.:1.347  
 Median :1.360  
 Mean   :1.362  
 3rd Qu.:1.370  
 Max.   :1.440  
[1] 0.02322725
> 
> 
$ R -q -e "x <- read.csv('egrep.log', header = F); summary(x); sd(x[ , 1])"
> x <- read.csv('egrep.log', header = F); summary(x); sd(x[ , 1])
       V1       
 Min.   :1.330  
 1st Qu.:1.340  
 Median :1.360  
 Mean   :1.365  
 3rd Qu.:1.380  
 Max.   :1.430  
[1] 0.02320288
> 
> 

I'm on the en_GB.utf8 locale, but the times are nearly indistinguishable.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.