Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Given the following files:

data/A/a.txt
data/B/b.pdf
...
date/P/whatever.log
...
data/Z/z.jpg

I would like to delete all files in the data/A/, data/B/, ..., data/Z/ directories except those files that are situated under one of the directories listed in the file data/dont_clean.txt. For example, if we have data/P listed in data/dont_clean.txt then nothing should be touched under data/P/, etc.

Something like:

find data/ -mindepth 2 -maxdepth 2 -type f -not -path {listed in data/dont_clean} -delete

Of course it is not a valid command.

I have also tried variants of

find data/ -mindepth 2 -maxdepth 2 -type f -exec grep data/dont_clean.txt '{}' \;

but I only created either an invalid command or I had no idea why I got the output I did.

I am using bash on Ubuntu 12.10

share|improve this question
    
Do you want the content of the files to be the same? Or is it you don't care, but files of the right name are immune, but can be changed wildly? –  kurtm Oct 12 '13 at 0:27
    
@kurtm Yes, I want the content to be unchanged. –  Ali Oct 12 '13 at 0:41
    
Sorry, wording confused me again. You want the content to be what it originally was, yes? –  kurtm Oct 12 '13 at 0:42
    
@kurtm Yes, exactly. And also the last modification date should be the same. It still allows you to move the file. However keep in mind that the subdirectories I am cleaning do have sub-subdirectories. I only sweeping out the 2nd level of the data/ directory tree, see the -mindepth 2 -maxdepth 2 flags I am passing to find. –  Ali Oct 12 '13 at 9:40

5 Answers 5

This is code that I only roughly tested but might layout an approach for you to take. Assuming you have a file, ignore.txt like this:

1/
2/

Sample data

And I had sample directories with files in them like this:

$ mkdir -p dirs/{1..5}
$ touch dirs/{1..5}/afile

Resulting in this:

$ tree dirs/
dirs/
|-- 1
|   `-- afile
|-- 2
|   `-- afile
|-- 3
|   `-- afile
|-- 4
|   `-- afile
`-- 5
    `-- afile

Example run

Now if we run this command against this tree:

$ find dirs/ -type f -print0 | fgrep -zFvf ./ignore.txt
dirs/5/afiledirs/4/afiledirs/3/afile

We can see that we're only getting back the files that are in directories not listed in ignore.txt.

So we can add a rm to the end to remove the non-excluded files.

$ find dirs/ -type f -print0 | fgrep -zFvf ./ignore.txt | xargs -0 rm -f

Checking we can see that it worked:

$ tree dirs/
dirs/
|-- 1
|   `-- afile
|-- 2
|   `-- afile
|-- 3
|-- 4
`-- 5

Problems to be worked out

One big problem with this approach is that the strings in the ignore.txt file might match other portions of the directory structure. So some care needs to be paid to making sure that the strings in this file are unique in the way that you expect.

Some blocking could be put around the strings so that they're anchored to the beginning or the end of the string to protect them.

Details

The above commands are doing the following:

  1. finding all the files under the directory dirs
  2. filtering out any files that are under a directory present in the igonre.txt file
  3. passing the filter list via xargs to the rm -f command
share|improve this answer
    
I can't say that I fully understand what is happening but I also the "one big problem." The solution doesn't have to be effective, it only needs to be bug free. How about writing a standalone shellscript that checks if a directory is in the ignore.txt file and running that in -exec? Seems easier to me. –  Ali Oct 11 '13 at 20:18
    
@Ali - that is surely an options as well. The only drawback is that if you have 1000 files, you'll be running the shell script 1000 times. The above approach will call rm only as many times as is required, since the xargs command will be putting as many paths as it can fit on the command line, prior to running. –  slm Oct 11 '13 at 20:21
    
@Ali - let me know if the details make it clearer. –  slm Oct 11 '13 at 20:23
    
Unfortunately, the new details doesn't solve the "one big problem." It is point 2. that is still unclear but further explanation won't help, please make it in such a way that that the filtering works as expected. As I said before, efficiency is not a primary concern. If I run a shellscript 1000 times, I run it 1000 times, I can live with it. However, doing bogus things accidentally is not acceptable. –  Ali Oct 11 '13 at 21:43

Sounds like a case for the comm command.

list of files to not delete in "keeper"
ls >current
comm -23 current ../keeper | more

check that it's the correct list

comm -23 current ../keeper | xargs rm
share|improve this answer
1  
Please re-read the question: directories are listed in the keeper file. –  Ali Oct 11 '13 at 21:23
1  
To the downvoter: please leave a comment. If mpez0 revises his answer, he has no chance to let you know about it so that you can revoke your downvote or even upvote his answer. –  Ali Oct 11 '13 at 23:17

xargs and find combination

Demo:

My Files :

[root@mail tmp]# find data/ -type f
data/A/d.txt
data/A/b.txt
data/A/a.txt
data/A/c.txt
data/B/e.txt
data/B/g.txt
data/B/f.txt
data/B/i.txt

Exclude List

[root@mail tmp]# cat exclude 
data/A/a.txt
data/B/e.txt

find with xargs

[root@mail tmp]# find data/ -type f $( xargs  -I{} echo -n " -not -path {} " < exclude )
data/A/d.txt
data/A/b.txt
data/A/c.txt
data/B/g.txt
data/B/f.txt
data/B/i.txt

Seems Output is OK, let delete but before that make sure output of following command is OK, then you can remove second last echo command.

[root@mail tmp]# find data/ -type f $( xargs  -I{} echo -n " -not -path {} " < exclude ) | xargs -n1 echo rm -rf
rm -rf data/A/d.txt
rm -rf data/A/b.txt
rm -rf data/A/c.txt
rm -rf data/B/g.txt
rm -rf data/B/f.txt
rm -rf data/B/i.txt
share|improve this answer
    
Thanks, so far this answer seems to be the closest to what I had in mind. In the exclude file directories are listed, under which nothing should be touched. As it stands, your solution doesn't work for directories unfortunately. Could you please adjust it? –  Ali Oct 11 '13 at 21:36

The only thing that needs to solved is to create a program that:

  • reads a line from the stdout,

  • then echos this line if it doesn't start with any of the strings listed in the data/dont_clean.txt file.

I couldn't solve this with a bash script easily so I wrote a little C++ program called my_program that does exactly this. Then I get what I want if I run:

find data/ -mindepth 2 -maxdepth 2 -type f | my_program | xargs rm -f 

However, I leave this question still open, I am still interested in a pure shellscript solution.

share|improve this answer

You could do it in two steps: mark (to make them distinctive) and then remove unprotected.

If all these files are from the same user, then you can use the list to change ownership or group to another user. Then use find to remove the rest and then change the user back. If all the files have the same read permissions (so you know how to switch back), you can for instance just remove read permission (to flag them) because it's much easier than changing users (no need for root privileges):

while read file; do chmod u-w -R "$file"; done < "data/dont_clean.txt"
find data/ -mindepth 2 -maxdepth 2 -type f -writable -delete
while read file; do chmod u+w -R "$file"; done < "data/dont_clean.txt"

Edited (added recursion) to protect whole directories. Changed from read to write permissions to avoid problems with chmod recursion (it couldn't read directory it just changed).

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.