Why is this

sed -e '/\s.*$/ s///' -e '/\(.*\)/ s//\L\1/' > filename.txt

faster than this?

sed -e 's/\s.*$//' -e 's/\(.*\)/\L\1/' > filename.txt

When I run them it seems that they do practically the same thing, but what really changes under the hood the makes one faster than the other?

UPDATE: Version: (GNU sed) 4.4 Input Data Size: 12GB

  • 3
    Yes, and how much faster? How did you test it? On what kind of data? – terdon May 14 '17 at 12:38

The first invocation of sed is applying the substitution commands (s) to a set of lines that are selected using a regular expression as an address range.

The second invocation applies the same substitutions as the first, but to all lines of the input data without using address ranges.

Testing on 225 MiB worth of email archives:

$ find . -type f -name "*.gz" -exec zcat {} + | time gsed -e '/\s.*$/ s///' -e '/\(.*\)/ s//\L\1/' >/dev/null
real    1m0,39s
user    0m49,69s
sys     0m10,53s

$ find . -type f -name "*.gz" -exec zcat {} + | time gsed -e 's/\s.*$//' -e 's/\(.*\)/\L\1/' >/dev/null
real    0m40,79s
user    0m34,02s
sys     0m7,85s

I ran this a few times. The timings presented are representative.

As you can see, I get the opposite results from what you claim to get. This may be due to the data. Similar results were had when testing OpenBSD sed on the same data (using slightly modified expressions since yours are GNU sed-specific), although the difference in timings were smaller.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.