1

I'm looking for a generalizable solution to this problem, 'cause I'm sure it will come up again.

I have a client that does medical research. Periodically we have to remove the datasets provided by the local health authority, and attest to the fact that we have securely done so.

We use rsnapshot with a retention of 

7 dailies, 
4 weeklies, 
12 monthlies, 
4 yearlies.

I recently needed to purge the rsnapshot tree of years-old data that looked a lot like this in the source location:

/path/to/studyName/data files yyyy mm 01/bunchOfFiles
/path/to/studyName/data files yyyy mm 02/bunchOfFiles
/path/to/studyName/data files yyyy mm 03/bunchOfFiles
/path/to/studyName/randomly placed and named data file 1
/path/to/studyName/randomly named admin file to keep 1
/path/to/studyName/randomly placed and named data file 2
/path/to/studyName/randomly placed and named data file 3
/path/to/studyName/randomly named admin file to keep 2
/path/to/studyName/randomly named admin file to keep 3
/path/to/studyName/randomly placed and named data file 4
/path/to/studyName/randomly named admin file to keep n
/path/to/studyName/randomly placed and named data file 5

Note that the ./data files* sub-dirs also occur in many other study locations, so they are not useful as find targets.

What I would have liked to have done would involve find with a couple of nested -exec commands. But find can't do that.

I tried nesting a -exec using bash -c within the first -exec command. After stubbing my toes on all sorts of things I needed to handle (like the spaces in the subdirectory names), my timer ran out, and it became more cost effective to just make a giant collection of rm commands to delete each instance of the file (or sub-directory containing data files).

for example, if it was possible to nest -execs, I think this would work:

find /path/to/rsnapshot_tree -type d -name studyName / 
  -exec 'find {}  -type d -iname "data files* -exec rm -rf {} \;'

OK - it can't work because of the ambiguity of the -exec termination and {}, but think of it as pseudo code

essentially,

  • find each instance of the directory "studyName" in the rshapshot tree, then apply rm -rf to the 3 directories described by "data files*"
  • Then repeat for each instance of misplaced data files in the
    studyName directory.

6 commands, instead of the >150 I had to copy, paste, edit and get right in a script.

Any suggestions about a better way would be much appreciated.

I have rsnapshot trees hosted on both linux (debian/ubuntu) and freeBSD hosts. Bash is always the shell (I change the default on the freeBSD hosts)

Improve this question
1
  • 1
    Not related to your question, but why retaining 4 weeklies and not 5? Check out the calendar, and see that sometimes you may want that 5th weekly backup/snapshot.
    – boardrider
    Jan 30, 2016 at 11:18

1 Answer 1

Reset to default
1

your question is not entirely clear to me, but if you have a dir structure as follows --

a
a/data
a/data/file1
a/studyName
a/studyName/data
a/studyName/data/file1
a/studyName/data/file2
a/studyName/data/file3

and you're looking for studyName/data/fileX, you could do --

find  . -path "*studyName/data*" -type f
./a/studyName/data/file1
./a/studyName/data/file2
./a/studyName/data/file3
Improve this answer
1
  • I'll have to try this on the next iteration of data removal. I suspect, given the human propensity for mis-filing, that we will be stuck with an approach like this -- gernerate a list of files, proof the list of files, use the proofed list to make a script to remove the files, then find empty directories and remove those. As an added bonus, the proofing process will also allow us to audit the tree for mis-placed copies of the data sets.
    – BISI
    Aug 15, 2020 at 17:44

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .