We are writing scripts for Linux systems, there has been some debate over what would be the most universally Linux present scripting language to use. Bash, SH, Posix? What?
migrated from stackoverflow.com Sep 25 '12 at 23:11
This question came from our site for professional and enthusiast programmers.
There are two programming environments that are available on every unix-like operating system, that are Turing-complete and that are able to call other programs: awk, and sh, the Bourne/POSIX shell family. AWK is oriented towards text processing (it complements more specialized utilities), while sh is oriented towards being a glue language to put programs together. Sh is the universal scripting language on Linux and across the unix world.
The POSIX standard defines mandatory features of sh itself and associated utilities. Most unix-like systems comply with POSIX 1003.1-2004 (a.k.a. Single Unix v3, a.k.a. the Open Group Base Specification issue 6); the latest version of that standard is POSIX 1003.1-2008 (a.k.a. Single Unix v4, a.k.a. the Open Group Base Specification Issue 7).
Every Linux and unix or Unix-like system has a Bourne-style shell at the path
Embedded Linux systems may have a stripped-down BusyBox system that doesn't implement all POSIX features. BusyBox has a large number of compile-time options to accommodate small-footprint systems, so it's difficult to know what to expect in advance, you have to tailor your scripts to a particular device. BusyBox is the most common small-footprint implementation of sh and assorted utilities; another one you might encounter is the extremely reduced shell environment in Android (later versions are less anemic).
Non-embedded Linux systems almost always have either dash or bash as
Non-embedded Linux systems almost always have Bash installed as
One of the characteristics of shell programming is that you aren't just using the
If you need portability beyond Linux, your best bet is to stick to POSIX. Other unix variants may not have bash installed (bash is part of the standard installation on OSX, but an optional package on *BSD and most commercial unices). Almost all unix variants other than Linux and OSX (i.e. *BSD and commercial unices) have some version of the Korn shell, at least pdksh. Many of bash's convenient extensions are from ksh, so it can be useful to write scripts that can run under both, but detecting where bash or ksh is located on an unknown system can be a bit of a pain.
The shell can't do everything. If you need a more sophisticated language, the two more common choices are Perl and Python (anything else is far behind as a unix scripting language). Perl is the traditional scripting language, and few non-embedded Linux systems lack it, but Python is gaining ground (boosted in part by being the recommended scripting language for Ubuntu). In the non-Linux world, Perl is part of the base installation on OSX and OpenBSD; it's optional but very commonly installed on FreeBSD, and optional but often installed on NetBSD.
In availability order:
After that, no one really cares since there isn't much you can't do with just those.
Ordinarily, I'd say
If you don't have to care about non-linux portability (and if you don't need to run on tiny distros or in embedded Linux devices like plastic-box routers), then you may as well make use of the significant enhancements that it offers over plain
Next would be
The only catch with perl is that there is also an enormous library of CPAN modules to do just about anything you can think of (and lots more that may never occur to you) - and not all of them will be available on every system with perl. They're usually of very high quality so it's easy to get into the habit of just using them - but if you do make use of them, you'll have to make sure they're installed. Many CPAN modules are pre-packaged for Linux, and the remainder are easily installed with the cpan tool (or
I'd like to be able to say
(Don't get the wrong idea from my sarcasm here - I really like python as a language, I just hate the fact that version and dependency management is a PITA. It's like going back 20+ years to the era of manually hunting for obscure bits of code and patches to get something compiled and running on your proprietary *nix)
I'd suggest follow ksh93 or even POSIX flavour and you can always use bash/zsh to run.
And Debian-based distro use mawk not gawk as default awk. So yes avoid gawk additions since mawk is way faster.
Not bash. Write to a close-to-POSIX sh like dash or ash. That's going to be most universal. I don't think there's anything else that's even a close competitor. (And this seems to me a factual question, not an "opinion" as one of the commentors complains.)
If you need something a bit more powerful than sh (for example, if you want real associative arrays), use awk. (Avoiding the gawk extensions. There are many versions of awk, but there's a largely shared core.) That should be available almost as broadly as sh is.
I would say the availability would rank somewhere in this order:
Although Python sounds like a good language, I've not used any OS that came with it in a base installation, though apparently Ubuntu does, maybe?
I believe the experts here have already provided great suggestions that should help you decide. The usability and availability of the different shells have been nicely described in the other posts.
On a different note, if I were you, I would also consider the goal that I want to achieve with the scripts. Every tool has its own merits and demerits. So, although I largely use Python, I won't use that in every case.
I can think of a few scenarios and mention some tools useful for them.
FTP files to-and-fro; process them; send email notifications; and so on
In this case it would be better to stick with the shell (e.g., Bash) and use the various utilities (e.g.,
Quick text processing
Once again, the shell. The utilities like
In the C/C++ domain, the universal tool for this is
Deployment and remote control
Either the shell or Python. For example,
Python, on the other hand, has a very useful module named
(Note that the suggestions above does not focus on the shell features much, but rather on the different utilities available.)
Perl is not in FreeBSD, NetBSD or DragonflyBSD base, sorry. OpenBSD has Perl in the base installation. OS X is nowadays a certified UNIX and may be thought as a some sort of BSD variant at some level and it has Perl, Python and Ruby in base.
Many proprietary UNIXen don't have Perl in their base, for example Solaris, AFAIK neither HP-UX or IBM AIX too...