50

This is my code

#!/bin/bash

showword() {
  echo $1
}

echo This is a sample message | xargs -d' ' -t -n1 -P2 showword

So I have a function showword which echoes whatever string you pass as a parameter to the function.

Then I have xargs trying to call the function and pass one word at a time to the function, and run 2 copies of the function in parallel. The thing that is not working is xargs doesn't recognize the function. How can I achieve what I am trying to do, how can I make xargs work with the function showword?

1

3 Answers 3

52

Try exporting the function, then calling it in a subshell:

showword() {
  echo $1
}

export -f showword
echo This is a sample message | xargs -d' ' -t -n1 -P2 bash -c 'showword "$@"' _

This causes xargs to execute

bash -c 'showword "$@"' _ This
bash -c 'showword "$@"' _ is
bash -c 'showword "$@"' _ a
            ︙

The arguments passed to the bash command are, well, passed into the bash environment, but starting from 0.  So, inside the function,

  • $0 is “_” and $1 is “This
  • $0 is “_” and $1 is “is
  • $0 is “_” and $1 is “a
  •       ︙

See Bash -c with positional parameters.

Note that export -f works only in Bash, and -Pn (--max-procs=max-procs) works only in GNU xargs.

8
  • 7
    One more question, why is the trailing _ required? Is it to stop xargs from processing anything else?
    – GMaster
    Commented Oct 1, 2014 at 6:51
  • 4
    @FazleA.: read unix.stackexchange.com/questions/152391/…
    – cuonglm
    Commented Oct 1, 2014 at 7:08
  • 2
    I am getting "export: Illegal option -f" when I do export -f Commented Nov 22, 2018 at 6:35
  • 1
    @cuonglm this does invoke a new bash process for every xargs item, right?
    – NarūnasK
    Commented May 10, 2019 at 19:42
  • 2
    Why does this work? Or, more specifically, why is the bash -c subshell needed (why can't xargs just detect and use the function just defined in the same shell process)?
    – Magnus
    Commented Mar 13, 2021 at 22:26
2

Just adding an alternative solution using parallel which I have started using in place of xargs. The task is much easier with parallel

#!/bin/bash

showword() {
  echo $1
}
export -f showword

parallel -j2 showword {} ::: This is a sample message
  • -j2 makes sure 2 copies of the function is run in parallel
  • ::: anything after this is passed as separate arguments to parallel, separation is whitespace
  • {} is replaced by the argument passed into parallel and fed into the showword function

If you are using zsh shell this solution will not work since zsh does not have any feature to export functions. You will need something like this:

#!/usr/bin/zsh

showword() {
  echo $1
}

# add the following to your .zshrc if you want env_parallel in your shell permanently
source /usr/bin/env_parallel.zsh

env_parallel -j2 --env showword showword {} ::: This is a sample message
2
  • Where does /usr/bin/env_parallel.zsh come from? I don't have it on my system... Commented Oct 19, 2022 at 21:18
  • 1
    @friederbluemle On Ubuntu, if I run the command dpkg -S /usr/bin/env_parallel.zsh, it says the file /usr/bin/env_parallel.zsh comes from the package parallel
    – GMaster
    Commented Oct 20, 2022 at 2:55
0

Let we have a function which takes argument and does anything (mine prints it two times)

$ twice() { echo $1$1 }

$ twice "hello"
hellohello

It doesn't work with pipe as it doesn't read any input

$ echo "hello" | twice
<nothing>

We can simply fix this by reading input to temp variable

$ echo "hello" | read s; twice $s
hellohello

Or with helper function

$ call() { read s; $1 $s }

$ echo "hello" | call twice
hellohello

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .