Is Unix find overwhelmed by multiple prunes and type f?

Question

I have spent days on this Rubik's cube. Anything I do to fix one problem breaks another.

I am on POSIX compliant MacOS X 10.5 thru 10.14. I am calling this from a Perl script in a context of

  system ("find blah blah > FILENAME");

I need Unix 'find' to do all these things at once.

• start at a volume root e.g. /Volumes/My HD
• do not cross file systems
• print files only, not directories or symlinks
• do not even descend into multiple directories like net dev system. (I.e. do not explore /Volumes/foo/dev/ but do explore /Volumes/foo/Users/Jim/dev/github/twonky/)
• the start point may contain spaces

Right now I am doing the following: (broken into several lines for readability; it's actually one long line)

 Find -x '/Volumes/foo/' 
    -path '/Volumes/foo//dev/*' -prune
    -path '/Volumes/foo//net/*' -prune
    -path '/Volumes/foo//system/*' -prune
    -o -type f -print

The reason for the double / is find’s printout includes the // because the starting point ends in a /. The Prune paths must agree, or they won't match. Why does the starting point end in /? Because if it doesn't, find fails on any starting point with a space in the name, like "My HD". Tried that.

Right now, find is only excludes the first directory in the list. The rest, it just ignores. I am currently testing on OS X 10.5 but I need something that works everywhere.

Is multiple prunes + files only + spaces in filenames a bridge too far? Am I just asking too much of find?

Answer 1

You need an "or" to accomplish the second match -- no single path will match both -path '/Volumes/foo//dev/*' and -path '/Volumes/foo//net/*'

Find -x '/Volumes/foo/' 
    \( -path '/Volumes/foo//dev/*' 
    -o -path '/Volumes/foo//net/*' 
    -o -path '/Volumes/foo//system/*' \) -prune
-o -type f -print

Answer 2

My answer with a pure Perl solution.

With this sandbox:

$ tree -F Volumes/ 
Volumes/ 
└── My\ HD/
    ├── Users/
    │   └── Jim/
    │       └── dev/
    │           └── github/
    │               └── twonky/
    │                   └── i_there.txt
    ├── dev/
    ├── net/
    ├── start.bat
    └── system/
        └── hello

9 directories, 3 files

The following Perl code using File::Find:

#!/usr/bin/perl

use strict;
use warnings;
use feature 'say';

use File::Find;

my $start = 'Volumes/My HD';
my $start_dev = (stat($start))[0];
my @exclude = qw/net dev system/;
my %skipdir;

sub wanted {
    my $name = $_;
    return if (stat($name))[0] != $start_dev;
    $skipdir{$File::Find::name} = 1 if $File::Find::dir eq $start && grep { $name eq $_ } @exclude;
    if (exists($skipdir{$File::Find::dir})) {
        $skipdir{$File::Find::name} = 1 if -d $name;
        return;
    }
    return if ! -f $name;
    say "Got: $File::Find::name";

}

my %args = (
    wanted => \&wanted,
    follow => 1,
    follow_skip => 1,
);


find(\%args, $start);

gives the expected (if I understood you right):

Got: Volumes/My HD/start.bat
Got: Volumes/My HD/Users/Jim/dev/github/twonky/i_there.txt

It is a POC, it can be enhanced.

Note also that you have the find2perl tool that is documented to be able to convert a specific find invocation to the associate Perl code using File::Find with the same criteria.

Now with Path::Class the code may seem simpler/easier to read (for the same result):

#!/usr/bin/perl

use strict;
use warnings;
use feature 'say';

use Path::Class;

my $start = Path::Class::Dir->new('Volumes/My HD');
my @exclude = qw/net dev system/;

$start->recurse(callback => sub {
    my $node = shift;
    if ($node->is_dir) {
        return $node->PRUNE if $node->parent eq $start && grep { $node->dir_list(-1) eq $_ } @exclude;
        return;
    }
    return $node->PRUNE if $node->stat()->dev != $start->stat()->dev;
    say 'Got: ', $node->stringify();
}, preorder => 1)

Answer 3

With your help, I was able to stabilize "find". However moving the code from OS X 10.5 to 10.10 broke it again. That was the last straw. 'find' is simply too obtuse, underdocumented and inconsistent, and it's a unix core feature for Pete's sake! This. This is why I hate other people's code. I started to hunker down to learn File::Find, then thought "what am I doing? I can code this myself in 20 minutes".

Which I summarily did.

sub iterate {
  my ($mydir, $ref_FH, $homevol, $ref_excludes) = @_;  # last is ref to hash

  return if (defined ($ref_excludes -> {$mydir}));   # No excludes

  my $thisvol = (stat($mydir))[0];    # What's my volume?
  return if ($thisvol != $homevol) ;  # No crossing volumes

  opendir (my $DIR, $mydir);
  while (defined (my $file = readdir($DIR))) {
    next if ($file eq '.' or $file eq '..');
    my $full = "$mydir/$file";   

    if (-l $full) {                                   # symlink
                                                         # nope
    } elsif (-f $full) {                              # file
      print {$$ref_FH} "$full\n";                        # print it
    } elsif (-d $full) {                              # dir
      &iterate($full, $ref_FH, $homevol, $ref_excludes); # iterate
    }
  }
}

And it's fast. And light - this code is half the size (and more maintainable) than the code that formatted "find"'s arg list!