What is the Linux command-line command that can identify such files?

AFAIK the find command (or grep) can only match a specific string inside the text file. But I want to match whole contents, i.e. I want to see which files match regular expression \0+, ignoring the line end character(s). Maybe the find . cat | grep idiom could work, but I don't know how to make grep ignoring lines (and treat the file as binary).

Background: Every few days, when my laptop freezes, my btrfs partition looses information: files opened for write gets their contents replaced with zeroes (the size of the file remains more-or-less intact). I use synchronization and I don't want these fake files to propagate: I need a way to identify them so I can grab them from backup.

  • you mean files having numeric zeros in it ? – Rahul Patil Dec 20 '12 at 11:08
  • 1
    I think it's about NULL characters rather than numeric zeroes. – gertvdijk Dec 20 '12 at 11:10
  • 8
    Let's take a step back here. Every few days, when your laptop freezes? Why aren't we trying to fix that, the real problem here? – D_Bye Dec 20 '12 at 11:32
  • 2
    @D_Bye that's good idea, but so far it didn't come too far:[unix.stackexchange.com/questions/57894/… – Adam Ryczkowski Dec 20 '12 at 13:23
  • 1
    have you considered the -v option to grep: filter out all files that have any byte 1 to 255. – ctrl-alt-delor Jun 1 '15 at 21:27

You can grep for ␀ characters using the Perl regex mode:

$ echo -ne "\0\0" > nul.bin
$ echo -ne "\0x\0" > non-nul.bin
$ grep -P "[^\0]" *.bin
Binary file non-nul.bin matches

So you can use this:

for path in *.foo
    grep -P "[^\0]" "$path" || echo "$path"
  • I get unexpected results, using GNU grep 2.5.4. Regardless of whether I use --binary-files=text or --binary-files=binary, it gives a true result for all non-empty data values, eg. "\0\0", "\0x\0", "abcd"... The exact code I used is: for typ in binary text ;do for dat in '\0\0' '\0x\0' 'abcd' '' ;do printf "$dat" >f; grep --binary-files=$typ -P '[^\0]' f >/dev/null && echo true || echo false; done; done – Peter.O Dec 21 '12 at 3:37
  • 1
    I have now further tried GNU grep) 2.10. This later version does give the expected results... so, a belated +1 – Peter.O Dec 21 '12 at 3:44
  • Fails on a file created with printf '\0\n\0\0\n\n' > file or printf '\n' > file for that matters. – Stéphane Chazelas Jun 1 '15 at 21:42
  • @StéphaneChazelas OP did say "ignoring the line end character(s)." So any file consisting of only \0 and \n characters (even zero of either) would be a match. – l0b0 Jun 1 '15 at 22:59

I agree with what D_Bye says about finding the root of the problem.

Anyway to check if a file only contains \0 and/or \n you could use tr:

<file tr -d '\0\n' | wc -c

Which returns 0 for null/newline and empty files.

  • 2
    tr -d '\0\n' solves the newline issue, which then only leaves the issue(?) of empty files being listed in the output... It does process every byte of every file though (which may or may not be an issue) +1 – Peter.O Dec 20 '12 at 13:02
  • @Peter.O: I missed the newline requirement, thank you. This solution isn't very optimized and if it is to run on a lot of data it would be better with a solution that moves on upon finding non-matching bytes. – Thor Dec 20 '12 at 16:39
  • It works very well. I my case I only had to make sure to exclude zero-length files. Thank you. – Adam Ryczkowski Dec 20 '12 at 19:21
  • 1
    This will also, however, count files with newlines in as being "empty". – Chris Down Dec 22 '12 at 11:52
  • @ChrisDown: I made the answer text clear as to what it does. It's not clear what the OP wants to do with newline-only files. – Thor Dec 24 '12 at 0:16

Here's a small python program that can do it:

import sys
def chunkCheck(fileObject, chunkSize=1024):
    while True:
        data = fileObject.read(chunkSize)
        if not data:
            return False
        if data.strip("\0"):
            return True

And in action:

$ printf '\0\0\0' > file
$ ./onlynulls file && echo "Only nulls" || echo "Non-null characters"
Only nulls
$ printf a >> file
$ ./onlynulls file && echo "Only nulls" || echo "Non-null characters"
Non-null characters

You can check multiple files by using find's -exec, xargs, GNU parallel, and similar programs. Alternatively, this will print filenames that need to be dealt with:

files=( file1 file2 )
for file in "${files[@]}"; do
    ./onlynulls "$file" || printf '%s\n' "$file"

Bear in mind that if you're going to pass the output of this to another program, filenames can contain newlines, so you should delimit it differently (fittingly, with \0).

If you have a lot of files, it would be better to use an option for parallel processing, since this only reads one file at a time.


I suspect those files are sparse, that is they don't have any disk space allocated to them, they just specify a file size (du would report 0 for them).

In which case, with GNU find, you could do (assuming no file path contains newline characters):

find . -type f -size +0 -printf '%b:%p\n' | grep '^0:' | cut -d: -f2-
  • Good point. I never thought about it. I'll try. Using du will prevent from scratching contents of every single file in the file system, so the whole procedure wouldn't take 30+ minutes to complete. – Adam Ryczkowski Dec 22 '12 at 9:58
  • (and printf %b above reports what du would report) – Stéphane Chazelas Dec 22 '12 at 16:19

Find files which contain only null-chars '\0' and newline chars '\n'.
The q in sed causes each file search to quit immediately upon finding any non-null character in a line.

find -type f -name 'file-*' |
  while IFS= read -r file ;do 
      out=$(sed -n '1=; /^\x00\+$/d; i non-null
                      ; q' "$file")
      [[ $out == "1" ]] &&  echo "$file"

Make test files

> file-empty
printf '%s\n' 'line1' 'line2' 'line3'      > file-with-text           
printf '%4s\n' '' '' xx | sed 's/ /\x00/g' > file-with-text-and-nulls
printf '%4s\n' '' '' '' | sed 's/ /\x00/g' > file-with-nulls-and-newlines
printf '%4s'   '' '' '' | sed 's/ /\x00/g' > file-with-nulls-only




Single file

Define the alias:

alias is_binary="python -c 'import sys; sys.exit(not b\"\x00\" in open(sys.argv[1], \"rb\").read())'"

Test it:

$ is_binary /etc/hosts; echo $?
$ is_binary `which which`; echo $?

Multiple files

Find all binary files recursively:

IS_BINARY='import sys; sys.exit(not b"\x00" in open(sys.argv[1], "rb").read())'
find . -type f -exec bash -c "python -c '$IS_BINARY' {} && echo {}" \;

To find all non-binary files, change && with ||.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.