I came across the terms awk and sed, awk goes once through all lines and performs a task whenever a line meets a certain condition, sed can manipulate a stream of input before it goes further to the output. I do not know personally for what purpose to use them, but i noticed they are referred to as powerfull even holy commands. Beside those two, what commands/applications have such a status?

closed as primarily opinion-based by muru, jordanm, glenn jackman, jimmij, Mikel Feb 12 '15 at 14:57

Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise. If this question can be reworded to fit the rules in the help center, please edit the question.

  • du, most definitely. Secretly an abbreviation for 'don't use'. – Simon Klaver Feb 12 '15 at 13:39
up vote 1 down vote accepted

To define "powerful" commands, one must first decide what this means.

The first requirement is universality. Some tools are very dedicated to one thing, yet can solve a very wide array of problems. gnuplot is a very basic plotting tool that is so good that even in scientific circles, you barely need anything else. And a single tool that can process regular expressions, can be used for almost anything, and that is indeed why sed is very powerful. You can do conversion between formats, data extraction, parsing, searching, replacing, and almost anything you can imagine. awk is similar, but also very very different: it's meant more for structured data, parsed line-by-line according to patterns, and is closer to more conventional programming languages (has more control flow and so on). Once you master these tools, you can solve any problem with them: both are turing complete, and can in principle solve any problem, even if it's unrelated to string processing - universality is the highest level of power: you can do anything. But you have to "want" to try and play with it. From this follows the next requirement.

The second component is legacy and tradition: commands that have existed for 40 years and have a huge codebase behind them, are also the most stable (long development), reliable (they can't stop developing them or change the functionality because 40 years of scripts would break), and there are a huge number of gurus out there that can do magic. This means that a tool that is widely known and a first choice for use by many people, is not necessarily the one that was the best at first, but the one that "stuck". However, if a lot of people use it, it's probably not bad, even though some things could be done better. The age is important here: python is extremely powerful as a language, but hasn't reached the status of the old gnu toolkits yet - it still gives a bit of a novelty feeling. bash is a golden standard not only because it's intuitive an powerful, but also because it's shipped as default shell almost everywhere and everybody uses it.

Cult status is very important here: people feel proud to be a part of some geek subculture. I can tell you first-hand that I felt really cool when I mastered emacs and latex, and also adopted hatred for vi almost immediately: partly because it really isn't an intuitive interface to me personally, but of course, this gets reinforced when you find out other people feel that way. What I'm trying to say, that a powerful tool is also powerful socially: a lesser known or poorly designed tool cannot rouse such strong emotions in people (neither negative nor positive - they are just bland). Some of the tools I mentioned have followers that could be compared to trekkies - not a bad achievement for a piece of software. awk could be thought as a predecessor to perl, and perl definitely has followers: most people agree that perl has terrible legibility and a lot of clumsy quirks. It's also the most powerful string-focused language out there, but without the cult status, most people might give up on it sooner, instead of playing around and trying to save the world and cook dinner with the same tool that was meant to chew through a bunch of web pages.

Finally, people like consistency, elegance and character... a tool can be powerful in terms of capabilites but nobody will want to use it because it just doesn't feel right. A powerful tool has to be intuitive at least to the people who like to use it. I'm not starting flamewars here, but surely most of the readers will find their own examples of unfortunate and clumsy design.

In short: well-designed for a well defined purpose, universal, large legacy and following, and ... a "soul". That would be my definition.

  • 1
    After over 20 years of using python (only the last 15 professionally), without knowing how to describe the feeling while doing so, I finally know: it's the novelty feeling. Thanks! – Anthon Feb 12 '15 at 15:06
  • Y'all gotta stop hating on perl. You don't get it. Perl was not "meant to chew through a bunch of web pages" -- it predates the internet. Perl 5 is a much more expressive language than the relatively conservative python 3. The the syntax is arcane, but so is lisp and C++. It owes much more to C and shell, with bits stolen from sed, than awk, but originality is its genius. You can do things in perl that would be considered absolute crimes in other OO languages. Meaning, it's not for everybody and everything, but perl is a work of art and pretty much defines powerful... – goldilocks Feb 13 '15 at 0:13
  • ...its design has had a profound impact on contemporary languages including python and I believe a wave of things yet to come ;] – goldilocks Feb 13 '15 at 0:14
  • -> whoops "predates the internet" should have been "predates the WWW". Don't want to sound like an ignoramus or anything... – goldilocks Feb 13 '15 at 0:22

One way to classify some 'old' commands - though not for 'more powerful' - is that they are builtin commands, meaning a command or a function, called from a shell, that is executed directly in the shell itself, instead of an external executable program which the shell would load and execute.
Examples include logout, cd, echo and history.

You can see if a command is a builtin with

builtin [command]

e..g

builtin echo

NO output means it is a builtin (uses the unix philosophy for nothing for ok)

If the command is not a builtin, e.g. ls then

builtin ls

returns an error -bash builtin: cp: not a shell builtin

As for 'most powerful' commands such as sed and awk, I'd actually remove that from the question as it seems very subjective and unlikely to result in a definitive argument.

  • 2
    The builtins are not "powerful" commands, just most basic and necessary. Shells override simple commands that are used frequently, and commands that are actually weak and the shell wants to extend their functionality. The real powerful commands would be very bad to include as a builtin, because the shell version would probably lag behind the original, and break most scripts. – orion Feb 12 '15 at 14:11
  • +1 Agreed, I thought that was fairly clear from my last paragraph but I've edited my first paragraph to make that clearer. Thanks! – Michael Durrant Feb 12 '15 at 16:28

Not the answer you're looking for? Browse other questions tagged or ask your own question.