24

I'm entering the world of Linux and at work I'm using grep more and more. By doing that I'm figuring out that sometimes it's not adequate for what I want.

I was struggling with grep a few days ago and a colleague of mine who is a senior Linux admin, told me to use awk. I was stunned by how fast I got a result.

So my question is when do you choose to use one over the other? What questions can I ask myself before going to work with grep and spending a lot of time, when I could have done it with awk and saved time?

Improve this question
3
  • This is a good question but it really is very broad and primarily opinion based. To answer your question, using grep as opposed to awk really depends on how comfortable you are using awk/sed. There are a few reasons why one would use this over the latter such as looking for something more specific than what grep has to offer or if you are attempting to replace/edit files with a specific string (using sed). But again, all depends on your comfort level and experience with using awk/sed.
    – ryekayo
    Aug 12, 2016 at 15:24
  • 11
    Use less to view a file. Use grep to search through a file. Use sed to edit a file. Use awk over grep and sed when the file you want to process has some kind of structure (such as columns). Use sed over awk when you mainly want to deal with lines (such as delete or add lines of text). I'm sure somebody will write a 20-pages answer that is more complete than mine.
    – Satō Katsura
    Aug 12, 2016 at 15:32
  • Hi Sato, It not about the length, but what is said. And you few lines are very informative. Thank you.
    – frankguthrie
    Aug 13, 2016 at 12:39

1 Answer 1

Reset to default
44

sed and awk are supersets of grep, there are things that are easier to do with one or the other.

grep foo can be written sed '/foo/!d' or awk /foo/, but consider:

grep -i foo would have to be sed '/[fF][oO][oO]/!d' unless you want to consider non-standard extensions like GNU's sed '/foo/I!d'. Or with awk: awk 'tolower($0) ~ /foo/' or again using a GNU extension: awk -v IGNORECASE=1 /foo/.

Things the different tools are good at and cumbersome with the other tools:

grep

grep is a simple tool but has very specialised modes of operation that are harder to reproduce with awk or sed:

  • grep -i for case insensitive matching (see above)
  • grep -Fe "$string" for fixed string search (export string; awk 'index($0, ENVIRON["string"])' with awk, no direct equivalent with sed).
  • (non standard) grep -r for recursive search
  • (non standard) grep -P/pcregrep for perl-like regexps (some sed implementations have perl-like regexp support though not the most major ones)
  • (non standard) grep -o to return the matched portion (several lines of awk or sed to do the same)
  • (non standard) grep -A/B/C to return context around the match (again painful to do in a similar fashion with sed or awk)

sed

  • s/foo/bar/: sed's s command has features that are hard to implement in awk like:
  • s/foo\(.*\)bar/\1/g: capturing (though GNU awk has a gensub() extension for that)
  • s/foo/bar/3: replace the 3rd occurrence on each line
  • (non-standard): in-place file editing (though it's also supported by GNU awk now).

awk

awk is the most feature rich of the three.

  • good for dealing with numbers
  • good for dealing with input formatted in columns.
  • good for extracting and combining data from different sources, with its associative arrays.

perl

perl as a practical extraction and reporting tool has the best of all those. That's what it was initially designed for (to be the tool that makes all those sed/awk obsolete).

Mastering perl to do text processing does give a serious advantage. I'd recommend spending some time on it, even before looking at the less common sed commands for instance.

performance

As a rule of thumb, the more specialised the tool, the most efficient it is at the task. But that also very much depends on the implementation, the task and a few other factors and performance can have trade-offs that may need to be taken into account.

For instance, there are some grep or sed implementations that are very fast, but for instance they don't support multibyte characters so can only work correctly on US-English text in multi-byte locales. Or they're fast because they work on a small fixed-length buffer and thus can't work on arbitrary input...

Improve this answer
3
  • Excellent answer! Do you know about the efficiency of the tools? I'm confused about the OP's statement, that awk is much faster than grep.
    – pfnuesel
    Aug 12, 2016 at 15:40
  • @pfnuesel, that very much depends on the implementation and on the usage pattern. For instance, mawk is known to be very efficient (but doesn't support multibyte characters which is part of the reason it's more efficient than some other tools)
    – Stéphane Chazelas
    Aug 12, 2016 at 15:50
  • 1
    @pfnuesel I think the OP is saying they were able to write the command they needed quickly with awk vs how long they had been struggling to write such a command with grep, not that the command they wrote executed faster with awk than grep.
    – Ed Morton
    Jun 24, 2021 at 12:46

Not the answer you're looking for? Browse other questions tagged or ask your own question.