Where are zsh and mksh incompatible with bash?

Question

To what extent can other POSIX-compatible shells function as reasonable replacements for bash? They don't need to be true "drop-in" replacements, but close enough to work with most scripts and support the rest with some modification.

1. I want to have explicit bash scripts - initscripts, DHCP client scripts, etc. - work with minimal modification

2. I want my own collection of more specialized shell scripts to not need too much modification

3. I want to have features like string manipulation and built-in regex pattern matching

The only serious contenders I know of are zsh and mksh. So, for those of you here who are good with either or both of them:

1. What features does bash have that zsh and mksh respectively do not?

2. What features do the shells share with bash, but use incompatible syntax for?

Answer 1

I'll stick to scripting features. Rich interactive features (command line edition, completion, prompts, etc.) tend to be very different, achieving similar effects in wholly incompatible ways. What features are in zsh and missing from bash, or vice versa? gives a few pointers on interactive use.

The closest thing to bash would be ATT ksh93 or mksh (the Korn shell and a clone). Zsh also has a subset of features but you would need to run it in ksh emulation mode, not in zsh native mode.

I won't list POSIX features (which are available in any modern sh shell), nor relatively obscure features, nor as mentioned above features for interactive use. Observations are valid as of bash 4.2, ksh 93u and mksh 40.9.20120630 as found on Debian wheezy.

Shell syntax

Quoting

$'…' (literal strings with backslash interpolation) is available in ksh93 and mksh. `$"…" (translated strings) is bash-specific.

Conditional constructs

Mksh and ksh93 have ;& to fall through in a case statement, but not ;;& to test subsequent cases. Mksh has ;| for that, and recent mksh allows ;;& for compatibility.

((…)) arithmetic expressions and [[ … ]] tests are ksh features. Some conditional operators are different, see “conditional expressions” below.

Coprocesses

Ksh and bash both have coprocesses but they work differently.

Functions

Mksh and ksh93 support the function name {…} syntax for function definitions in addition to the standard name () {…}, but using function in ksh changes scoping rules, so stick to name () … to maintain compatibility. The rules for allowed characters in function names vary; stick to alphanumerics and _.

Brace expansion

Ksh93 and mksh support brace expansion {foo,bar}. Ksh93 supports numeric ranges {1..42} but mksh doesn't.

Parameter expansion

Ksh93 and mksh support substring extraction with ${VAR:offset} and ${VAR:offset:length}, but not case folding like ${VAR^}, ${VAR,}, etc. You can do case conversion with typeset -l and typeset -u in both bash and ksh.

They support replacement with ${VAR/PATTERN/STRING} or ${VAR/PATTERN//STRING}. The quoting rules for STRING are slightly different, so avoid backslashes (and maybe other characters) in STRING (build a variable and use ${VAR/PATTERN/$REPLACEMENT} instead if the replacement contains quoting characters).

Array expansion (${ARRAY[KEY]}, "${ARRAY[@]}", ${#ARRAY[@]}, ${!ARRAY[@]}) work in bash like in ksh.

${!VAR} expanding to ${OTHERVAR} when the value of VAR is OTHERVAR (indirect variable reference) is bash-specific (ksh does something different with ${!VAR}). To get this double expansion in ksh, you need to use a name reference instead (typeset -n VAR=OTHERVAR; echo "$VAR"). ${!PREFIX*} works the same.

Process substitution

Process substitution <(…) and >(…) is supported in ksh93 but not in mksh.

Wildcard patterns

The ksh extended glob patterns that need shopt -s extglob to be activated in bash are always available in ksh93 and mksh.

Mksh doesn't support character classes like [[:alpha:]].

IO redirection

Bash and ksh93 define pseudo-files /dev/tcp/HOST/PORT and /dev/udp/HOST/PORT, but mksh doesn't.

Expanding wildcards in a redirection in scripts (as in var="*.txt"; echo hello >$a writing to a.txt if that file name is the sole match for the pattern) is a bash-specific feature (other shells never do it in scripts).

<<< here-strings work in ksh like in bash.

The shortcut >& to redirect syntax errors is also supported by mksh but not by ksh93.

Conditional expressions

[[ … ]] double bracket syntax

The double bracket syntax from ksh is supported by both ATT ksh93 and mksh like in bash.

File operators

Ksh93, mksh and bash support the same extensions to POSIX, including -a as an obsolete synonym of -e, -k (sticky), -G (owned by egid), -O (owner by euid), -ef (same file), -nt (newer than), -ot (older than).

-N FILE (modified since last read) isn't supported by mksh.

Mksh doesn't have a regexp matching operator =~. Ksh93 has this operator, and it performs the same matching as in bash, but doesn't have an equivalent of BASH_REMATCH to retrieve matched groups afterwards.

String operators

Ksh93 and mksh support the same string comparison operators < and > as bash as well as the == synonym of =. Mksh doesn't use locale settings to determine the lexicographic order, it compares strings as byte strings.

Other operators

-v VAR to test if a variable is defined is bash-specific. In any POSIX shell, you can use [ -z "${VAR+1}" ].

Builtins

alias

The set of allowed character in alias names isn't the same in all shells. I think it's the same as for functions (see above).

builtin

Ksh93 has a builtin called builtin, but it doesn't execute a name as a built-in command. Use command to bypass aliases and functions; this will call a builtin if one exists, otherwise an external command (you can avoid this with PATH= command error_out_if_this_is_not_a_builtin).

caller

This is bash-specific. You can get a similar effect with .sh.fun, .sh.file and .sh.lineno in ksh93. In mksh there's at last LINENO.

declare, local, typeset

declare is a bash-specific name for ksh's typeset. Use typeset: it also works in bash.

Mksh defines local as an alias for typeset. In ksh93, you need to use typeset (or define an alias).

Mksh has no associative arrays (they're slated for an as yet unreleased version).

I don't think there's an exact equivalent of bash's typeset -t (trace function) in ksh.

cd

Ksh93 doesn't have -e.

echo

Ksh93 and mksh process the -e and -n options like in bash. Mksh also understands -E, ksh93 doesn't treat it as an option. Backslash expansion is off by default in ksh93, on by default in mksh.

enable

Ksh doesn't provide a way to disable builtin commands. To avoid a builtin, look up the external command's path and invoke it explicitly.

exec

Ksh93 has -a but not -l. Mksh has neither.

export

Neither ksh93 nor mksh has export -n. Use typeset +x foo instead, it works in bash and ksh.

Ksh doesn't export functions through the environment.

let

let is the same in bash and ksh.

mapfile, readarray

This is a bash-specific feature. You can use while read loops or command substitution to read a file and split it into an array of lines. Take care of IFS and globbing. Here's the equivalent of mapfile -t lines </path/to/file:

IFS=$'\n'; set -f
lines=($(</path/to/file))
unset IFS; set +f

printf

printf is very similar. I think ksh93 supports all of bash's format directives. mksh doesn't support %q or %(DATE_FORMAT)T; on some installations, printf isn't an mksh builtin and calls the external command instead.

printf -v VAR is bash-specific, ksh always prints to standard output.

read

Several options are bash-specific, including all the ones about readline. The options -r, -d, -n, -N, -t, -u are identical in bash, ksh93 and mksh.

readonly

You can declare a variable as read-only in Ksh93 and mksh with the same syntax. If the variable is an array, you need to assign to it first, then make it read-only with readonly VAR. Functions can't be made read-only in ksh.

set, shopt

All the options to set and set -o are POSIX or ksh features.

shopt is bash-specific. Many options concern interactive use anyway. For effects on globbing and other features enabled by some options, see the section “Options” below.

source

This variant of . exists in ksh as well. In bash and mksh, source searches the current directory after PATH, but in ksh93, it's an exact equivalent of ..

trap

The DEBUG pseudo-signal isn't implemented in mksh. In ksh93, it exists with a different way to report information, see the manual for details.

type

In ksh, type is an alias for whence -v. In mksh, type -p does not print the path to the executable, but a human-readable message; you need to use whence -p COMMAND instead.

Options

shopt -s dotglob — don't ignore dot files in globbing

To emulate the dotglob option in ksh93, you can set FIGNORE='@(.|..)'. I don't think there's anything like this in mksh.

shopt -s extglob — ksh extended glob patterns

The extglob option is effectively always on in ksh.

shopt -s failglob — error out if a glob pattern matches nothing

I don't think this exists in either mksh or ksh93. It does in zsh (default behavior unless null_glob or csh_null_glob are set).

shopt -s globstar — **/ recursive globbing

Ksh93 has recursive globbing with **/, enabled with set -G. Mksh doesn't have recursive globbing.

shopt -s lastpipe — run the last command of a pipeline in the parent shell

Ksh93 always runs the last command of a pipeline in the parent shell, which in bash requires the lastpipe option to be set. Mksh always runs the last command of a pipeline in a subshell.

shopt -s nocaseglob, shopt -s nocasematch — case-insensitive patterns

Mksh doesn't have case-insensitive pattern matching. Ksh93 supports it on a pattern-by-pattern basis: prefix the pattern with ~(i).

shopt -s nullglob — expand patterns that match no file to an empty list

Mksh doesn't have this. Ksh93 supports it on a pattern-by-pattern basis: prefix the pattern with ~(N).

Variables

Obviously most of the BASH_xxx variables don't exist in ksh. $BASHPID can be emulated with the costly but portable sh -c 'echo $PPID', and has been recently added to mksh. BASH_LINE is .sh.lineno in ksh93 and LINENO in mksh. BASH_SUBSHELL is .sh.subshell in ksh93.

Mksh and ksh93 both source the file given in ENV when they start up.

EUID and UID don't exist in ksh93. Mksh calls them USER_ID and KSH_UID; it doesn't have GROUPS.

FUNCNAME and FUNCNEST don't exist in ksh. Ksh93 has .sh.fun and .sh.level. Functions declared with function foo { …; } (no parentheses!) have their own name in $0.

GLOBIGNORE exists in ksh93 but with a different name and syntax: it's called FIGNORE, and it's a single pattern, not a colon-separated list. Use a @(…|…) pattern. Ksh's FIGNORE subsumes bash's, with a wholly different syntax.

Ksh93 and mksh have nothing like HOSTTYPE, MACHTYPE and OSTYPE. Nor SHELLOPTS or TIMEFORMAT.

Mksh has PIPESTATUS, but ksh93 doesn't.

Mksh and ksh93 have RANDOM.

Answer 2

This question is rather too broad.

Both mksh and zsh are shells that support a lot of GNU bash-specific extensions, but there are always some that are not understood.

zsh supports more stuff, but only in its native zsh mode, which is not compatible with POSIX shells (such as GNU bash, AT&T ksh93, mksh). Also, mksh is much leaner and faster and more portable.

In general, if this is your scripts we’re talking about, go ahead, just test them. (mksh does not support bash4-style associative arrays yet. The “declare” command is bash-specific, “typeset” is an equivalent. I am not familiar enough with zsh to state anything about that outright. ksh93 does not have “local” but also uses “typeset” for that.) But if this is about, say, running a bash-less Debian system, forget it. The existence of bash is part of the “promise” (API/ABI of the system), and much relies on it.

Disclaimer: I’m the mksh developer.

Answer 3

ZSH Comparison of Shells

In recent years there has been a certain amount of crossover in the extensions, however. Zsh (as of 3.1.6) has bash's ${var/old/new}' feature for replacing the text old with the text new in the parameter $var. Note one difference here: while both shells implement the syntax${var/#old/new}' and ${var/%old/new}' for anchoring the match of old to the start or end of the parameter text, respectively, in zsh you can't put the#' or %' inside a parameter: in other words{var/$old/new}' where old begins with a #' treats that as an ordinary character in zsh, unlike bash. To do this sort of thing in zsh you can use (from 3.1.7) the new syntax for anchors in any pattern,(#s)' to match the start of a string, and `(#e)' to match the end. These require the option EXTENDED_GLOB to be set.

I'm most unfamiliar with mksh, so I don't know where to look for that answer.

If you are looking for a secure replacement for shell, none of these shells differ much from Bash as far as the inherent flaw you are discussing.

A language like Perl handles inputs more securely. But maintainability is also key here. The Perl shell replacement is not very well adopted. It's the responsibility of the shell maintainer to handle inputs securely. So when you write scripts, validate, validate, validate everything! Use code contracts to assure correct results every time!

Perl Shell

FSF Statement on Shell Shock

Answer 4

One feature not enabled by default in mksh is shell history.

• In your .mkshrc just set:

  export HISTFILE=~/.mksh-history