3

If a find is searching a large disc, and is aborted for some reason, I'd like to be able to start it again, from where it left off.

Is there a version of 'find' that does this?

The problem is that, if you save the directory, and start it from there, it won't do the same find. For example:

find /bigvol -type f -execdir whereami="$(pwd)";do_it {} \; echo whereami >/bigvol/state

Will put the currently searched directory into the file /bigvol/state, let's say it was /bigvol/down/in/the/deep

However, if you start the next find there:

find "$(cat /bigvol/state)" -type f -execdir whereami="$(pwd)";do_it {} \; echo whereami >/bigvol/state

Then you'll find everything in 'deep' (and re-execute 'deep' for the first few files in that directory), but you'd then stop and never search /bigvol/further

I'd like to be able to say something like:

find <dir>:<starting from>

So, if I saved the last file worked on in , in the above example, start it again.

Ideally the syntax would allow:

find <dir>:<starting_from>    -- start from the last file and re-execute for that filefind <dir>:+<starting_from>    -- start from the file after the last file last file
filefind <dir>:-<starting_from>    -- start from the file before the last file last file

Any ideas?

  • 1) short answer is no, it depend too much of the problem your are solving while going down directory structure with find predicate. however a long answer might be yes, this involve storing directory and files on a temporary file before executing any statement. – Archemar Dec 7 '15 at 8:50
  • 2) this looks like an Y part of XY problem (meta.stackexchange.com/questions/66377/what-is-the-xy-problem/…), what is value of X ? – Archemar Dec 7 '15 at 8:52
  • Yes, Archemar, I understand your point about the XY problem. I use 'find' quite a bit. Recently, I've been using it as part of the process of removing duplicates in my system. So I've been looking at all the files on the system. That's a lot. If it gets interrupted, it has to start again. It's annoying of it takes several days. I'm building a 'locate' directory as a possible work around to this, but, I'd the next comment interruptable – Peter Brooks Dec 7 '15 at 10:13
  • #!/bin/bash # # To find duplicate files # for i in /Volumes/* do if [ -d $i ] then cd $i find -x "$(pwd)" -type f -size +5k -exec shasum "last_dir/{}" \; >>/Volumes/vol01/checksums_of_all_files_on_system & fi done – Peter Brooks Dec 7 '15 at 10:13
  • Can you assume that the directory tree hasn't changed at all? If it's changed, the program would need to remember not just where it left off but also everything that it's seen. Even if you only cared about what was present initially, the traversal order could change, e.g. because some file name search tree was rebalanced. – Gilles Dec 7 '15 at 21:49
0

I don't know of any way to do this. Maybe another tool can do it. Or maybe you need another way to look at the situation.

You could split up the find action into multiple finds. The /bigvol folder probably has subfolders. You can split up the action into a find for each of those folders. If it has processed the first several folders, you can skip those later on.

The problem here is that one of those subfolders can have 95% of all files. You could analyze this with du. If you know that, you can split it up further. The question is how far you go with this approach.

  • 1
    Thank you. Yes, I'm doing that already: for i in dir1 dir dir2 do find $i.... I'd like to, though, actually start from the next file. – Peter Brooks Dec 7 '15 at 10:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.