I'm trying to write a simple bash function that takes, as its arguments, a number of files and/or directories. It should:

  1. Fully qualify the filenames.
  2. Sort them.
  3. Remove duplicates.
  4. Print all that actually exist.
  5. Return the number of non-existent files.

I have a script that almost does what I want, but falls down on the sorting. The return value of the script as it stands is correct, but the output is not (unsorted and duplicates). If I uncomment the | sort -u statement as indicated, the output is correct but the return value is always 0.

N.B. Simpler solutions to solve the problem are welcome but the question is really about why is this occurring in the code I have. That is, why does adding the pipe seemingly stop the script incrementing the variable r?

Here's the script:

function uniqfile
    local r=0 

    for arg in "$@"
        readlink -e "$arg" || (( ++r ))

    done #| sort -u    ## remove that comment

    return $r
  • Just a small observation. You can reduce for arg in "$@" to for arg. "If 'in WORDS ...;' is not present, then 'in "$@"' is assumed." - help for
    – manatwork
    Commented Sep 30, 2011 at 9:35

2 Answers 2


This is a well known bash pitfall, due to this feature:

Each command in a pipeline is executed as a separate process (i.e., in a subshell).

so that modified variables are local to the subshell, and not visible once back in the parent.

To avoid that, rephrase your code to avoid the pipeline, with a process substitution:

 for arg in "$@"
        readlink -e "$arg" || (( ++r ))

    done > >(sort -u)
  • Thankyou. That's great. I wonder if you could tell me the name of the >(..command..) construct. I think I know how it works but feel I should do some further reading.
    – tjm
    Commented Sep 30, 2011 at 9:07
  • 2
    @tjm: it is called process substitution
    – enzotib
    Commented Sep 30, 2011 at 9:15
  • Process substitution in Bash has many forms: tldp.org/LDP/abs/html/process-sub.html
    – slm
    Commented Sep 15, 2014 at 11:58
  • Process substitution is a form of inter-process communication that allows the input or output of a command to appear as a file. The command is substituted in-line, where a file name would normally occur, by the command shell. This allows programs that normally only accept files to directly read from or write to another program. Commented May 20, 2015 at 18:24

The | sort -u forces the preceding bit (so the whole for loop) to run in a sub-process (bash needs a 'STDOUT' to redirect into the sort 'STDIN'. (Internet seems to think ksh and bash handle this case slightly differently .. first or last command in the pipe sequence gets put into a subshell?)

This thread goes over a similar problem, and has a neat solution at the end: http://ubuntuforums.org/showthread.php?t=312017

    exec 3< <(du | sort -n)  

    while read size dir; do
      [ $size -gt 1000 ] && ((n++))
    done <&3
    exec 3<&-

    echo "Found $n too big files"

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .