4

I am trying to write a script that will apply a bash function for each file in a directory recursively. For example if the directory tests had all my files and sub-directories in it, the script

find tests -type f -print0 | xargs -0 echo

Works perfectly. Now I want to be able to apply a bash function rather than just echo, so I came up with something like this:

function fun() {
    echo $1
}

export -f fun

find tests -type f -print0 | xargs -0 bash -c 'fun "$@"'

However this only outputs a single file in tests, where before it output all the files. I would expect these two to run the same no?

  • 1
    You can try the exec option of find, i think will do the job. stackoverflow.com/questions/5119946/… – George Vasiliou Nov 22 '16 at 0:44
  • Is your function even defined in that bash process?  If you say bash -c, you get a non-interactive shell, which doesn't look at .bashrc.  Secondly, does your function look at multiple arguments?  Thirdly, you should say bash -c 'fun "$@"' sh, because the first word after the command string after bash -c is assigned to $0, and therefore isn't included in "$@". … … … … … … … … … … … … … … … … OK, ignore the first point; I hadn't noticed that you had exported the function. – G-Man Nov 22 '16 at 0:50
  • 1
    And I hadn't noticed that you showed us the function!  (D'oh! facepalm!)  My second point is that, (as you might have noticed from your experiment with echo) xargs executes the command with all the files at once (or, at least, as many as can be passed on one command line — this is in the thousands) and not once per command.  Since your function prints only $1 (and not all the arguments), you get only one. – G-Man Nov 22 '16 at 1:04
2

Use a shell script.

To quote from this excellent answer:

Do other programs besides your shell need to be able to use it? If so, it has to be a script.

As already noted in other answers, it may be possible to work around this by exporting the function, but it's certainly a lot simpler to just use a script.

#!/bin/bash
# I'm a script named "fun"!  :)
printf '%s\n' "$@"

Then you just put the fun script somewhere in your PATH (perhaps in ~/bin), make sure it's executable, and you can run:

find tests -type f -exec fun {} +
  • Nice, I guess this is how I check all ruby files syntax: find . -name "*.rb" -exec ruby -c {} \; – Nakilon Aug 29 '18 at 9:29
1

There two ways to solve this issue: you could either tell xargs to run command with only one argument at a time using -n1 or loop for all arguments inside your function fun:

function fun() { echo $1 }
export -f fun
find tests -type f -print0 | xargs -n1 -0 bash -c 'fun "$@"' --

or

function fun() { while [ -n "$1" ]; do echo $1; shift; done }
export -f fun
find tests -type f -print0 | xargs -0 bash -c 'fun "$@"' --
  • 1
    Maybe a clearer way to loop through arguments is with a for: function fun() { for a; do echo "$a"; done } – trosos Nov 22 '16 at 3:12
  • 1
    @trosos, or just printf '%s\n' "$@", since printf is better than echo. – Wildcard Nov 22 '16 at 23:44
0

If you use GNU Parallel it looks like this:

function fun() {
    echo "$@"
}
export -f fun

find tests -type f -print0 | parallel -0 -Xj1 fun

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.