For performance reasons, I want to use find to list a large number of files, but also include a counter on each line. The following is what I have so far:

local root_dir="."
local ouptut_file="/tmp/foo.txt"

File_Number=99              ## Initial value

echo "" > "${ouptut_file}"  ## Initialize file to empty

find "$root_dir"   \
    -type f        \
    -depth 2       \
    -name '*.xxx'  \
    -print0        \
| sort -z          \
| xargs -0  printf "DoSomething %5d '%s'\n" $[File_Number++] -- \
> "${ouptut_file}"

This outputs

DoSomething    99 '--'
DoSomething     0 'dirA/dirB/file1.xxx'
DoSomething     0 'dirA/dirB/file2.xxx'
DoSomething     0 'dirA/dirB/file3.xxx'

what I want is

DoSomething     100 'dirA/dirB/file1.xxx'
DoSomething     101 'dirA/dirB/file2.xxx'
DoSomething     102 'dirA/dirB/file3.xxx'

How do I include the counter and also eliminate the first spurious line in the output.

System: Am using bash on macOS Ventura 13.0.1

  • You never need to initialize a file. You can just do find ... > file and the file will be created if it doesn't exist and will be emptied and its contents replaced with the output of the command if it did exist.
    – terdon
    Nov 30, 2022 at 12:10
  • ... or if the noclobber option is set (with set -o noclobberin ~/.bashrc), use find ... >| file which will clobber any already existing file upon redirecting output to them, something practical but unsafe by design ...
    – Cbhihe
    Nov 30, 2022 at 18:21
  • 1
    The easiest way to number lines is to use the nl command like that find ... | nl --starting-line-number=100. It doesn't support zero terminated strings and also has no way of prepending output lines with custom text.
    – legolegs
    Nov 30, 2022 at 19:09

2 Answers 2


A couple of notes:

  1. $[...] is a really old form of arithmetic expansion. These days the bash documentation doesn't even mention it. Use $((...)) instead.
  2. xargs will start a new process for each invocation of the command, so any changes to variables in one won't affect other invocations.
  3. For printf(1), -- is just like any another argument. What's happening is that xargs runs printf "DoSomething %5d '%s'\n" $[File_Number++] dirA/dirB/file1.xxx dirA/dirB/file2.xxx ..., and printf loops over the arguments to fit the format string requirements. So every alternate filename is actually being treated as a number, resulting in the 0 in the output.

You might want to use something like:

find "$root_dir"   \
    -type f        \
    -depth 2       \
    -name '*.xxx'  \
    -print0        \
| sort -z          \
| while read -r -d '' file; do
     printf "DoSomething %5d '%s'\n" $((File_Number++)) "$file"
  done > "${ouptut_file}"

Also note that if you're actually using this to print commands for running in a shell, you should use %q instead of '%s' for proper quoting. From help printf:

In addition to the standard printf(1) formats, %b means to expand backslash escape sequences in the corresponding argument, and %q means to quote the argument in a way that can be reused as shell input.

  • Using printf %q is not the safest if it's to be interpreted by a Bourne-like shell. As the OP's on macos, they have the option to use zsh which has more reliable quoting operators. Nov 30, 2022 at 9:34
  • @StéphaneChazelas is right about the safety of printf %q, but if you can always know the conditions of your file system and can be assured that you'll never be sending it anything risky, that would mitigate the risk.
    – Br.Bill
    Nov 30, 2022 at 19:35
  • 1
    Point 2 isn't relevant here, because the variable here is expanded by the shell before xargs sees anything. So there's only ever one increment of $File_number regardless of how many processes are started there. Otherwise, a great explanation of why the script doesn't do what Peter was expecting. Dec 1, 2022 at 9:08

I'd use perl:

find "$root_dir"   \
    -type f        \
    -depth 2       \
    -name '*.xxx'  \
    -print0        \
| sort -z          \
| perl -l -0ne 'printf "something %05d %s\n", $., $_' > "$output_file"

If you need to quote the files, putting ' on either side may not be enough depending on what file names may contain and what language those quotes are intended for. For instance, in several languages including some shells, the \ and newline characters are still special within quotes and would need to be escaped. In any case, the ' character cannot be embedded asis in a single quoted string.

If the quoting is for a Bourne-like shell, you can do quoting with a:

sub shquote {return "'" . ($_[0] =~ s/'/'\\''/gr).  "'"}

function which is the safest form of quoting in those shells¹. So here:

perl -l -0ne '
  sub shquote {return "'\''" . ($_[0] =~ '"s/'/'\\\\''/gr"').  "'\''"}
  printf "something %05d %s\n", $., shquote($_)'

Or if you struggle to keep track of all those quotes:

perl -l -0nse '
  sub shquote {return $q . ($_[0] =~ s/$q/$q\\$q$q/gr) . $q}
  printf "something %05d %s\n", $., shquote($_)' -- -q="'"

Also note that outputting to files with fixed name in world writable areas is bad practice from a security standpoint. Someone could create a symlink with the same name to some sensitive file you have write access to for instance. Creating files in /tmp should only be done with randomly generated file names and with O_NOFOLLOW like mktemp does, or in freshly made private directories there (bailing out if it can't be created, like for instance if it already existed).

¹ Safer than the String::ShellQuote CPAN module in any case which has a few issues.

  • I am getting No such file or directory on the line with the perl statement. I am using the bash shell that comes with Mac so that is probably the issue. Dec 1, 2022 at 0:59

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .