Let's say I have a simple stupid script that removes files by ending, looking like this:

rm *.uvw *.xyz

The script, or rm, to be precise, writes messages on stderr if it cannot find at least one file with the specified ending.

Now let's say the script is a bit bigger and does a bit more with a bit more file types and I'm not interested in which file types exist and which don't, but the complains about non-existent file types obstruct the rest of the output and error messages I am more interested in, so I want to filter the output:

rm *.uvw *.xyz 2>&1 | grep -v "No such file or directory"

This works fine for the most part, but it removes the message part of interactive dialogs, which for example ask, if a write-protected file should be deleted, so I get prompted without the according message.

I do not understand this behaviour and could not find any related information. Can someone explain this?

  • This is probably a buffering issue but I don't have time to test it out right now. I'd suggest trying to solve it with tee. – Julie Pelletier Aug 7 '16 at 16:39
  • 3
    This seems like an XY problem - why not set the shell's nullglob option appropriately? – steeldriver Aug 7 '16 at 16:45
up vote 10 down vote accepted

The problem

When rm prompts the use for input, it does not put a newline at the end of the prompt:

$ rm *.uvw *.xyz
rm: remove write-protected regular empty file 'a.xyz'?

grep is line-based. It can only process complete lines. It cannot tell whether the line should be printed until the line is complete. Thus, standard utilities for dealing with buffering, such as stdbuf, cannot help.

The solution

Use nullglob and remove the missing files messages.

Without nullglob, the messages you don't want appear:

$ rm *.uvw *.xyz
rm: cannot remove '*.uvw': No such file or directory
rm: remove write-protected regular empty file 'a.xyz'? n

With it, the "No such file or directory" message is suppressed:

$ shopt -s nullglob
$ rm *.uvw *.xyz
rm: remove write-protected regular empty file 'a.xyz'? n

Refinement

If there is no file at all that matches either glob, then a different error message appears:

$ shopt -s nullglob
$ rm *.uvw *.xyz
rm: missing operand
Try 'rm --help' for more information.

A simple way to avoid this is to make sure that at least one such file exists:

shopt -s nullglob
[ -e "deleteme.xyz" ] ||touch deleteme.xyz
rm *.uvw *.xyz

Since deleteme.xyz is going to be erased anyway, there is no harm in touching it before we run rm.

  • Small inelegance of this solution I've stumbled upon just now: If none of the patterns matches, it prints an rm: missing operand to stderr, which is actually correct, but will look like a bug if one does not know exactly how the script works. – Wanderer Aug 8 '16 at 15:33
  • 1
    @N.Nord I just updated the answer with a fix for that special case. – John1024 Aug 9 '16 at 5:31
  • Thank you for expanding your answer! At the risk of getting esoteric but for the sake of perfectionists: For the unlikely case that deleteme.xyz already exists and is write-protected, touch fails and prints an according message, so I added an [ -e "deleteme.aux" ] || right infront of the touch command. Should be at least bulletproof now. – Wanderer Aug 9 '16 at 13:54
  • 1
    @N.Nord Good observation! Answer updated with -e test. – John1024 Aug 9 '16 at 16:59

If you're using GNU grep, look at the '-s' option to suppress messages.

-s, --no-messages
    Suppress error messages about nonexistent or unreadable files.

*** Note: this is not a portable option.

  • Oh, yes, this is indeed preferable over what I've tried at first. However, to my understanding, the problem with prompt messages will persist with this solution, due to the reason John1024 lined out at the beginning of his answer. – Wanderer Aug 9 '16 at 20:19

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.