How to find files with 100% NUL characters in their contents?

Question

What is the Linux command-line command that can identify such files?

AFAIK the find command (or grep) can only match a specific string inside the text file. But I want to match whole contents, i.e. I want to see which files match regular expression \0+, ignoring the line end character(s). Maybe the find . cat | grep idiom could work, but I don't know how to make grep ignoring lines (and treat the file as binary).

Background: Every few days, when my laptop freezes, my btrfs partition looses information: files opened for write gets their contents replaced with zeroes (the size of the file remains more-or-less intact). I use synchronization and I don't want these fake files to propagate: I need a way to identify them so I can grab them from backup.

Answer 1

You can grep for ␀ characters using the Perl regex mode:

$ echo -ne "\0\0" > nul.bin
$ echo -ne "\0x\0" > non-nul.bin
$ grep -P "[^\0]" *.bin
Binary file non-nul.bin matches

So you can use this:

for path in *.foo
do
    grep -P "[^\0]" "$path" || echo "$path"
done

Answer 2

I agree with what D_Bye says about finding the root of the problem.

Anyway to check if a file only contains \0 and/or \n you could use tr:

<file tr -d '\0\n' | wc -c

Which returns 0 for null/newline and empty files.

Answer 3

Here's a small python program that can do it:

import sys
def chunkCheck(fileObject, chunkSize=1024):
    while True:
        data = fileObject.read(chunkSize)
        if not data:
            return False
        if data.strip("\0"):
            return True
sys.exit(chunkCheck(open(sys.argv[1])))

And in action:

$ printf '\0\0\0' > file
$ ./onlynulls file && echo "Only nulls" || echo "Non-null characters"
Only nulls
$ printf a >> file
$ ./onlynulls file && echo "Only nulls" || echo "Non-null characters"
Non-null characters

You can check multiple files by using find's -exec, xargs, GNU parallel, and similar programs. Alternatively, this will print filenames that need to be dealt with:

files=( file1 file2 )
for file in "${files[@]}"; do
    ./onlynulls "$file" || printf '%s\n' "$file"
done

Bear in mind that if you're going to pass the output of this to another program, filenames can contain newlines, so you should delimit it differently (fittingly, with \0).

If you have a lot of files, it would be better to use an option for parallel processing, since this only reads one file at a time.

Answer 4

I suspect those files are sparse, that is they don't have any disk space allocated to them, they just specify a file size (du would report 0 for them).

In which case, with GNU find, you could do (assuming no file path contains newline characters):

find . -type f -size +0 -printf '%b:%p\n' | grep '^0:' | cut -d: -f2-

Answer 5

Find files which contain only null-chars '\0' and newline chars '\n'.
The q in sed causes each file search to quit immediately upon finding any non-null character in a line.

find -type f -name 'file-*' |
  while IFS= read -r file ;do 
      out=$(sed -n '1=; /^\x00\+$/d; i non-null
                      ; q' "$file")
      [[ $out == "1" ]] &&  echo "$file"
  done

Make test files

> file-empty
printf '%s\n' 'line1' 'line2' 'line3'      > file-with-text           
printf '%4s\n' '' '' xx | sed 's/ /\x00/g' > file-with-text-and-nulls
printf '%4s\n' '' '' '' | sed 's/ /\x00/g' > file-with-nulls-and-newlines
printf '%4s'   '' '' '' | sed 's/ /\x00/g' > file-with-nulls-only

output

./file-with-nulls-and-newlines
./file-with-nulls-only

Answer 6

if [ -z $(`od -a | grep "[!0\n]"`) ]; then echo "Only zeros"; fi

Edit: I could swear that the title set by OP sad "100% Zeroes" And i didn't notice the binary part, try od to get raw binary data from a file?

Logic: output anything but \x0|\n, if len(output) > 0: Then it contains something other then 100% zeroes.

I don't see why this is a bad idea to code in bash, since i assume it's a bash script the OP is looking for?