A few times when I read about programming I came across the "callback" concept.

Funnily, I never found an explanation I can call "didactic" or "clear" for this term "callback function" (almost any explanation I read seemed to me enough different from another and I felt confused).

Is the "callback" concept of programming existent in Bash? If so, please answer with a small, simple, Bash example.

  • See stackoverflow.com/questions/5672289 for a similar question. – dsstorefile1 Sep 21 at 6:01
  • 2
    Is "callback" an actual concept or is it "first-class function"? – Cedric H. Sep 22 at 9:13
  • You may find declarative.bash interesting, as a framework that explicitly leverages functions configured to be invoked when a given value is needed. – Charles Duffy Sep 22 at 14:48
  • Another relevant framework: bashup/events. Its documentation includes a lot of simple demos of callback use, such as for validation, lookups, etc. – PJ Eby Sep 22 at 17:52
  • 1
    @CedricH. Voted for you. "Is "callback" an actual concept or is it "first-class function"? " is a good question to ask as another question? – prosody-Gab Vereable Context Sep 23 at 0:01
up vote 42 down vote accepted
+150

In typical imperative programming, you write sequences of instructions and they are executed one after the other, with explicit control flow. For example:

if [ -f file1 ]; then   # If file1 exists ...
    cp file1 file2      # ... create file2 as a copy of a file1
fi

etc.

As can be seen from the example, in imperative programming you follow the execution flow quite easily, always working your way up from any given line of code to determine its execution context, knowing that any instructions you give will be executed as a result of their location in the flow (or their call sites’ locations, if you’re writing functions).

How callbacks change the flow

When you use callbacks, instead of placing the use of a set of instructions “geographically”, you describe when it should be called. Typical examples in other programming environments are cases such as “download this resource, and when the download is complete, call this callback”. Bash doesn’t have a generic callback construct of this kind, but it does have callbacks, for error-handling and a few other situations; for example (one has to first understand command substitution and Bash exit modes to understand that example):

#!/bin/bash

scripttmp=$(mktemp -d)           # Create a temporary directory (these will usually be created under /tmp or /var/tmp/)

cleanup() {                      # Declare a cleanup function
    rm -rf "${scripttmp}"        # ... which deletes the temporary directory we just created
}

trap cleanup EXIT                # Ask Bash to call cleanup on exit

If you want to try this out yourself, save the above in a file, say cleanUpOnExit.sh, make it executable and run it:

chmod 755 cleanUpOnExit.sh
./cleanUpOnExit.sh

My code here never explicitly calls the cleanup function; it tells Bash when to call it, using trap cleanup EXIT, i.e. “dear Bash, please run the cleanup command when you exit” (and cleanup happens to be a function I defined earlier, but it could be anything Bash understands). Bash supports this for all non-fatal signals, exits, command failures, and general debugging (you can specify a callback which is run before every command). The callback here is the cleanup function, which is “called back” by Bash just before the shell exits.

You can use Bash’s ability to evaluate shell parameters as commands, to build a callback-oriented framework; that’s somewhat beyond the scope of this answer, and would perhaps cause more confusion by suggesting that passing functions around always involves callbacks. See Bash: pass a function as parameter for some examples of the underlying functionality. The idea here, as with event-handling callbacks, is that functions can take data as parameters, but also other functions — this allows callers to provide behaviour as well as data. A simple example of this approach could look like

#!/bin/bash

doonall() {
    command="$1"
    shift
    for arg; do
        "${command}" "${arg}"
    done
}

backup() {
    mkdir -p ~/backup
    cp "$1" ~/backup
}

doonall backup "$@"

(I know this is a bit useless since cp can deal with multiple files, it’s only for illustration.)

Here we create a function, doonall, which takes another command, given as a parameter, and applies it to the rest of its parameters; then we use that to call the backup function on all the parameters given to the script. The result is a script which copies all its arguments, one by one, to a backup directory.

This kind of approach allows functions to be written with single responsibilities: doonall’s responsibility is to run something on all its arguments, one at a time; backup’s responsibility is to make a copy of its (sole) argument in a backup directory. Both doonall and backup can be used in other contexts, which allows more code re-use, better tests etc.

In this case the callback is the backup function, which we tell doonall to “call back” on each of its other arguments — we provide doonall with behaviour (its first argument) as well as data (the remaining arguments).

(Note that in the kind of use-case demonstrated in the second example, I wouldn’t use the term “callback” myself, but that’s perhaps a habit resulting from the languages I use. I think of this as passing functions or lambdas around, rather than registering callbacks in an event-oriented system.)

First it's important to note that what makes a function a callback function is how it's used, not what it does. A callback is when code that you write is called from code that you didn't write. You're asking the system to call you back when some particular event happens.

An example of a callback in shell programming is traps. A trap is a callback that isn't expressed as a function, but as a piece of code to evaluate. You're asking the shell to call your code when the shell receives a particular signal.

Another example of a callback is the -exec action of the find command. The job of the find command is to traverse directories recursively and process each file in turn. By default, the processing is to print the file name (implicit -print), but with -exec the processing is to run a command that you specifies. This fits the definition of a callback, although at callbacks go, it is not very flexible since the callback runs in a separate process.

If you implemented a find-like function, you could make it use a callback function to call on each file. Here's an ultra-simplified find-like function that takes a function name (or external command name) as argument and calls it on all regular files in the current directory and its subdirectories. The function is used as a callback which is called every time call_on_regular_files finds a regular file.

shopt -s globstar
call_on_regular_files () {
  declare callback="$1"
  declare file
  for file in **/*; do
    if [[ -f $file ]]; then
      "$callback" "$file"
    fi
  done
}

Callbacks aren't as common in shell programming as in some other environments because shells are primarily designed for simple programs. Callbacks are more common in environments where data and control flow are more likely to move back and forth between parts of the code that are written and distributed independently: the base system, various libraries, the application code.

  • 1
    Particularly nicely explained – roaima Sep 21 at 9:03
  • 1
    @JohnDoea I think the idea is that it's ultra-simplified in that it's not a function that you'd really write. But perhaps an even simpler example would be something with a hard-coded list to run the callback on: foreach_server() { declare callback="$1"; declare server; for server in 192.168.0.1 192.168.0.2 192.168.0.3; do "$callback" "$server"; done; } which you could run as foreach_server echo, foreach_server nslookup, etc. The declare callback="$1" is about as simple as it can get though: the callback has to be passed in somewhere, or it's not a callback. – IMSoP Sep 21 at 10:42
  • 3
    'A callback is when code that you write is called from code that you didn't write.' is simply wrong. You can write a thing that does some non-blocking async work, and run it with a callback it will run when completed. Nothing is related to who wrote the code, – mikemaccana Sep 21 at 11:43
  • 4
    @mikemaccana Of course it's possible that the same person wrote the two parts of the code. But it isn't the common case. I'm explaining the basics of a concept, not giving a formal definition. If you explain all the corner cases, it's difficult to convey the basics. – Gilles Sep 21 at 11:49
  • 1
    Glad to hear it. I disagree that people writing both the code that uses a callback and the callback isn't common or is an edge case, and, because of the confusion, that this answer conveys the basics. – mikemaccana Sep 21 at 12:05

"callbacks" are just functions passed as arguments to other functions.

At shell level, that simply means scripts / functions / commands passed as arguments to other scripts / functions / commands.

Now, for a simple example, consider the following script:

$ cat ~/w/bin/x
#! /bin/bash
cmd=$1; shift
case $1 in *%*) flt=${1//\%/\'%s\'};; *) flt="$1 '%s'";; esac; shift
q="'\\''"; f=${flt//\\/'\\'}; p=`printf "<($f) " "${@//\'/$q}"`
eval "$cmd" "$p"

having the synopsis

x command filter [file ...]

will apply filter to each file argument, then call command with the outputs of the filters as arguments.

For instance:

x diff zcat a.gz b.bz   # diff gzipped files
x diff3 zcat a.gz b.gz c.gz   # same with three-way diff
x diff hd a b  # hex diff of binary files
x diff 'zcat % | sort -u' a.gz b.gz  # first uncompress the files, then sort+uniq them, then compare them
x 'comm -12' sort a b  # find common lines in unsorted files

This is very close to what you can do in lisp (just kidding ;-))

Some people insist on limiting the "callback" term to "event handler" and/or "closure" (function + data/environment tuple); this is by no way the generally accepted meaning. And one reason why "callbacks" in those narrow senses aren't of much use in shell is because pipes + parallelism + dynamic programming capabilities are so much more powerful, and you're already paying for them in terms of performance, even if you try to use the shell as a clunky version of perl or python.

  • While your example looks pretty useful, it's sufficiently dense that I'd have to really pick it apart with the bash manual open to figure out how it works (and I've worked with simpler bash most days for years.) I never did learn lisp. ;) – Joe Sep 22 at 11:15
  • 1
    @Joe if it's OK to work with only two input files and no % interpolation in filters, the whole thing could be reduced to: cmd=$1; shift; flt=$1; shift; $cmd <($flt "$1") <($flt "$2"). But that's a lot less useful and illustrative imho. – mosvy Sep 22 at 11:25
  • 1
    Or even better $1 <($2 "$3") <($2 "$4") – mosvy Sep 22 at 11:31
  • +1 Thanks. Your comments, plus staring at it and playing with the code for some time, clarified it for me. I also learned a new term, "string interpolation", for something I've been using forever. – Joe Sep 24 at 4:33

Kind of.

One simple way to implement a callback in bash, is to accept the name of a program as a parameter, which acts as "callback function".

# This is script worker.sh accepts a callback in $1
cb="$1"
....
# Execute the call back, passing 3 parameters
$cb foo bar baz

This would be used like this:

# Invokes mycb.sh as a callback
worker.sh mycb.sh

Of course you don't have closures in bash. Hence, the callback function doesn't have access to the variables on the caller side. You can, however, store data the callback needs in environment variables. Passing information back from the callback to the invoker script is trickier. Data could be placed into a file.

If your design allows that everything is handled in a single process, you could use a shell function for the callback, and in this case the callback function has of course access to the variables on the invoker side.

Just to add a few words to the other answers. The function callback operates on function(s) external to the function that calls back. For this to be possible either a whole definition of the function to be called back needs to be passed to the function calling back, or its code should be available to the function calling back.

The former (passing code to another function) is possible, though I'll skip an example for this would involve complexity. The latter (passing the function by name) is a common practice, as the variables and functions declared outside of one function's scope are available in that function as long as their definition precedes the call to the function that operates on them (which, in turn, as to be declared before it's called).

Also note, that a similar thing happens when functions are exported. A shell that imports a function may have a framework ready and be just waiting for function definitions to put them in action. Function export is present in Bash and caused previously serious problems, btw (that was called Shellshock):

I'll complete this answer with one more method of passing a function to another function, which is not explicitly present in Bash. This one is passing it by address, not by name. This can be found in Perl, for example. Bash offers this way neither for functions, nor variables. But if, as you state, you want to have a wider picture with Bash as just an example, then you should know, that the function code may reside somewhere in the memory, and that code may be accessed by that memory location, which is called its address.

One of the simplest example of callback in bash is one a lot of people are familiar with but don't realise what design pattern they are actually using:

cron

Cron allows you to specify an executable (a binary or script) that the cron program will call back when some conditions are met (the time specification)

Say you have a script called doEveryDay.sh. The non-callback way to write the script is:

#! /bin/bash
while true; do
    doSomething
    sleep $TWENTY_FOUR_HOURS
done

The callback way to write it is simply:

#! /bin/bash
doSomething

Then in crontab you'd set something like

0 0 * * *     doEveryDay.sh

You would then not need to write the code to wait for the event to trigger but instead rely on cron to call your code back.


Now, consider HOW you would write this code in bash.

How would you execute another script/function in bash?

Let's write a function:

function every24hours () {
    CALLBACK=$1 ;# assume the only argument passed is
                 # something we can "call"/execute
    while true; do
        $CALLBACK ;# simply call the callback
        sleep $TWENTY_FOUR_HOURS
    done
}

Now you've created a function that accepts a callback. You can simply call it like this:

# "ping" google website every day
every24hours 'curl google.com'

Of course, the function every24hours never returns. Bash is a bit unique in that we can very easily make it asynchronous and spawn a process by appending &:

every24hours 'curl google.com' &

If you don't want this as a function you can do this as a script instead:

#every24hours.sh
CALLBACK=$1 ;# assume the only argument passed is
               # something we can "call"/execute
while true; do
    $CALLBACK ;# simply call the callback
    sleep $TWENTY_FOUR_HOURS
done

As you can see, callbacks in bash is trivial. It is simply:

CALLBACK_SCRIPT=$3 ;# or some other 
                    # argument to 
                    # function/script

And calling the callback is simply:

$SOME_CALLBACK_FUNCTION_OR_SCRIPT

As you can see form above, callbacks are rarely directly features of languages. They are usually programming in a creative manner using existing language features. Any language that can store a pointer/reference/copy of some code block/function/script can implement callbacks.

  • Other examples of programs/scripts that accept callbacks include watch and find (when used with -exec parameter) – slebetman Sep 23 at 8:22

A callback is a function called when some event occurs. With bash, the only event handling mechanism in place is related to signals, the shell exit, and extended to shell errors events, debug events and function/sourced scripts return events.

Here is an example of a useless but simple callback leveraging signal traps.

First create the script implementing the callback:

#!/bin/bash

myCallback() {
    echo "I've been called at $(date +%Y%m%dT%H%M%S)"
}

# Set the handler
trap myCallback SIGUSR1

# Main loop. Does nothing useful, essentially waits
while true; do
    read foo
done

Then run the script in one terminal:

$ ./callback-example

and on another one, send the USR1 signal to the shell process.

$ pkill -USR1 callback-example

Each signal sent should trigger the display of lines like these ones in the first terminal:

I've been called at 20180925T003515
I've been called at 20180925T003517

ksh93, as shell implementing many features that bash later adopted, provides what it calls "discipline functions". These functions, not available with bash, are called when a shell variable is modified or referenced (i.e. read). This open the way to more interesting event driven applications.

For example, this feature allowed X11/Xt/Motif style callbacks on graphic widgets to be implemented in an old version of ksh that included graphic extensions called dtksh. See dksh manual.

Only Perl-example available:

In the Perl example, we create a for_each function that takes a reference to an array and an anonymous function. Next, we go through the array, and perform an anonymous function (closure), passing to it as a parameter the next element of the array.

When using the for_each function, we first define the $ serminal variable $ initialized by zero. Then we pass a reference to the array and a closure function to the for_each function, in which we add each element of the array to the $ sum variable. After executing the for_each function, $ sum will contain the sum of the array.

sub for_each {
 my($arr, $cb) = @_;
 for my $item (@$arr) {
   $cb->($item);
 }
}

my $sum = 0;
for_each [3, 4, 2, 21, 13, 666], sub {
 $sum += $_[0];
};

ptint $sum;

As the for_each example shows, the callback function is the transfer of executable code as one of the parameters of another code. Often, the transmitted function works as a closure, i.e. has access to lexical variables and can be defined in other program code contexts and not available for direct calling from the parent function.


In fact, callback is an analog of function polymorphism, namely, it allows you to create functions of a more general purpose rather than creating a series of functions that are the same in structure but differ only in individual places with executable subtasks.


  • 6
    The fact that you don't give an example in bash is worth mentioning and explaining... (Also, test run your code). – Kusalananda Sep 21 at 5:56
  • @Kusalananda bash is very complicated for me.) Sorry. – Dillinger è morto Sep 21 at 6:00
  • 7
    Then why are you trying to answer a question that is explicitly, only about bash? – Charles Duffy Sep 22 at 14:47

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.