All the target files have been deleted. Of course, when I try to run any deletes again, the files aren't there to delete. Sorry to take your time.

I'm using bash on cygwin.

I have the output of fdupes in a file. I'm grepping the output to exclude a directory I want to keep intact, and wanting to delete the rest of the files listed.

I have some entries with spaces:

./NewVolume/keep/2009/conference/conference/conference 004.jpg

Which trips up xargs:

$ cat real-dupes.txt |xargs rm {}
rm: cannot remove ‘2009/conference/conference/conference’: No such file or directory`

When I try the -0 switch, it looks like the lines get globbed together:

$ cat real-dupes.txt |xargs -0 rm
xargs: argument line too long

Other questions have answers where the asker is adviced to use find to feed the arguments into xargs. That's not helpful in my scenario, because I don't believe that I can easily use find to identify the duplicates I want to get rid of. Also, the fdupes job ran some 12+ hours, so I really want to use this data set.

As far as I know, fdupes cannot exclude a directory from its automated delete, so I can't use it out of the box, either.

closed as off-topic by user394, Scott, vonbrand, mdpc, Jeff Schaller Dec 30 '15 at 1:45

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "Questions describing a problem that can't be reproduced and seemingly went away on its own (or went away when a typo was fixed) are off-topic as they are unlikely to help future readers." – Scott, vonbrand, mdpc, Jeff Schaller
If this question can be reworded to fit the rules in the help center, please edit the question.

  • If it's just spaces you should be able to handle it with: sed 's/ /\\ /' real-dupes.txt | xargs rm {}. This won't handle the many other special characters that can be present in filenames, though; for some examples: *$<tab>"' See this question for more details. – Wildcard Dec 29 '15 at 23:04
  • @Wildcard I got the same errors with your sed solution: rm: cannot remove ‘2009/photos’: No such file or directory -- there may be other characters that are problematic, but the majority of the errors thrown are from this apparent space. Pretty sure I used the normal space bar to name these files, although some originated on NTFS filesystems. – user394 Dec 29 '15 at 23:07
  • I mis-diagnosed the problem I was experiencing and all the answers are superfluous. – user394 Dec 29 '15 at 23:13
  • So, what, the sed solution deleted the files that remained and had spaces in them? – Wildcard Dec 29 '15 at 23:20
  • That could be, but I tried a number of things with the same error set (Can't delete files), so I'm not sure which one was the first to get it right. – user394 Dec 29 '15 at 23:43

xargs -0 is not working because it expects a null-terminated string, which it does not find, therefore reading up all the input and globbing its argument.

Just convert every new line (I suppose there is a filename per line) with a \0, like this:

cat real-dupes.txt | tr '\n' '\0' | xargs -0 rm {}
  • :S getting this: rm: cannot remove ‘./win7/Documents/keep/2009_movies/zoo 010.avi’: No such file or directory – user394 Dec 29 '15 at 23:09
  • And there is such file in the directory? =) – Kira Dec 29 '15 at 23:10
  • 1
    ... no ... they've apparently been deleted by previous attempts :P – user394 Dec 29 '15 at 23:12

By default xargs splits at whitespace. You may be able to use the non-standard -0 option to split at character \000, but your input has to be prepared to match the expectation. (find ... -print0 is one of the ways of doing this - assuming your version of find has the -print0 option.)

Provided that none of your files contains a new line in their name (i.e. you have one file per line) you can use xargs like this:

xargs -I{} rm {} <real-dupes.txt

The man page for xargs has this to say about the -I flag:

-I replace-str Replace occurrences of replace-str in the initial-arguments with names read from standard input. Also, unquoted blanks do not terminate input items; instead the separator is the newline character. Implies -x and -L 1.

  • The -0 option to xargs is a non-standard extension, as is the -print0 extension to find. – Andrew Henle Dec 29 '15 at 23:25
  • 1
    @AndrewHenle the OP had first suggested -0 so it seemed reasonable to reference it in my answer. – roaima Dec 29 '15 at 23:59
  • 1
    It is perfectly reasonable, and quite appropriate, but I think for completeness it's also important to note non-standard extensions so someone who runs across your answer in the future doesn't wonder what's wrong with their version xargs. – Andrew Henle Dec 30 '15 at 0:52

Not the answer you're looking for? Browse other questions tagged or ask your own question.