2

This command works fine for dir with "regular" names as dir1 mydir my-dir, etc

ls | parallel 'echo -n {}" "; ls {}|wc -l'

give me the number of files for each dir

but for dir with white spaces like "my dir" or my long dir name don't work and give error.

How to quote/escape the white spaces?

1
  • 5
    You can start by not parsing the output of ls and using another tool. Commented Nov 21, 2021 at 14:22

4 Answers 4

3

GNU Parallel has no problems dealing with spaces:

$ mkdir 'a  b'
$ touch 'a  b/c  d'
$ ls | parallel 'echo -n {}" "; ls {}|wc -l'
a  b 1

It requires -0, if the names contain \n:

$ mkdir 'a

b'
$ touch 'a

b/c

d'
# fails
$ ls | parallel 'echo -n {}" "; ls {}|wc -l'
$ parallel 'echo -n {}" "; ls {}|wc -l' ::: *
# works
$ printf "%s\0" * | parallel -0 'echo -n {}" "; ls {}|wc -l'
$ parallel -0 'echo -n {}" "; ls {}|wc -l' ::: *

So what you see is probably due to your ls doing something weird: It may be an alias to ls --some-weird-option. Try \ls instead (or use printf ... | ... -0 or -0 ... ::: * as show above):

\ls | parallel 'echo -n {}" "; ls {}|wc -l'

(PS: Are you aware of --tag:

parallel --tag 'ls {}|wc -l' ::: *

)

2

Without the complications of parallel, xargs, or passing pathnames back and forth:

shopt -s nullglob dotglob

for dir in */; do
    set -- "$dir"/*
    printf '%s:\t%s\n' "${dir%/}" "$#"
done

That is, iterate over all directories and count the number of names that the * glob expands to in each.

The above is for bash, and we set the nullglob shell option to ensure that non-matching patterns are removed instead of retained unexpanded. I'm also setting the dotglob shell option to be able to match hidden names.

The zsh shell can do it while at the same time filtering the glob matches for only directories (for the loop) and only, e.g., regular files (for the loop body). In the code below, the glob qualifier (ND/) makes the preceding * only match directories, with the same effect as with nullglob and dotglob set in the bash shell, and (ND.) makes the preceding * only match regular files in the same way.

for dir in *(ND/); do
    set -- $dir/*(ND.)
    printf '%s:\t%s\n' $dir $#
done

Would you want to do this recursively, to get the count of names in each directory in a hierarchy, then you could just plug in the above in find:

find . -type d -exec bash -O nullglob -O dotglob -c '
    for dir do
        set -- "$dir"/*
        printf "%s:\t%s\n" "$dir" "$#"
    done' bash {} +

(the above is a bit different from the plain bash loop at the start of this answer as this would never count names in directories accessed via symbolic links), or,

find . -type d -exec zsh -c '
    for dir do
        set -- $dir/*(ND.)
        printf "%s:\t%s\n" $dir $#
    done' zsh {} +
1
  • See also for dir (*(ND/)) () { printf '%s:\t%s\n' $dir $#; } $dir/*(ND.oN) which avoid clobbering the positional parameter list. Commented Nov 21, 2021 at 19:06
1

Assuming that you only want to list the number of regular files in the sub-directories from where you execute your cmd, here is a one-liner that will do that for you:

 $ find . -maxdepth 1 -type d ! -name "." -print0 2>/dev/null \
   | xargs -0 -I {} sh -c 'printf "%20s:  %d\n" "{}" "$(find "{}" -maxdepth 1 -type f 2>/dev/null| wc -l)"'

Example output:

              ./Maildir:  0
             ./.dvisvgm:  0
               ./.pyenv:  5
             ./.ipython:  0
   ./.ipynb_checkpoints:  3
                ./.tmux:  1
         ./.virtualenvs:  12
         ./seaborn-data:  2
               ./.local:  2
                ./bgpix:  12
                 ./.vim:  7
     ...
  • I added 2>/dev/null in each find block only to avoid some unwanted file access issues on the platform I used to run tests. You may do away with it if you foresee no such file permission issues when stat'ing your files as part of your find cmds.
  • I also suppress all output concerning $PWD (your present working directory, denoted .) in keeping with my assumption stated above, which was that you are only interested in counting regular files in current 1st-level subdirectories.
  • to count regular files in your whole sub-directory tree, starting at $PWD, just omit the -maxdepth 1 global option in the first find block above (but keep it in the 2nd one).

To better highlight the similarity with the solution relying on parallel (below), the above can be rewritten:

$ xargs -0 -I {} sh -c 'printf "%20s:  %d\n" "{}" "$(find "{}" -maxdepth 1 -type f 2>/dev/null| wc -l)"' \
  < <(find . -maxdepth 1 -type d ! -name "." -print0 2>/dev/null)

Relying on parallel instead of xargs, as in the above, requires escaping some quotes as follows (output is exactly as before):

$ parallel -0 -I {} \
  'sh -c "printf \"%20s: %d\n\" \"{}\" \"$(find {} -maxdepth 1 -type f 2>/dev/null | wc -l)\""' \
  :::: < <(find -maxdepth 1 -type d ! -name "." -print0 2>/dev/null)
  • xargs and parallel use the same two arguments -0 -I {} ,
  • the shell command to execute is the Bourne shell between single quotes,
    'sh -c "printf ..."', and
  • the input to parallelize, introduced by ::::, is the process substitution input file <(...) which contains the output of the "find all subdirectories at first sublevel starting at $PWD" cmd.
0

A workaround found using find instead of ls.

find * -type d -maxdepth 0 | parallel 'echo -n {}" "; ls {}|wc -l'
1
  • 1
    Hi @elbarna ! Careful with using global options such as -maxdepth 0after stat options such as the argument -type d. Orders matters here and global options always come first in find. Also maybe my solution is not what you are looking for... Apparently you want to count files AND sub-directories ONLY in your non-hidden sub-directories ? At least it is what this solution does. Finally favor printf over echowhenever you can for all these reasons.
    – Cbhihe
    Commented Nov 21, 2021 at 16:37

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .