So I recovered a text file from an old hdd, but I failed to completely recover all of the data. The data that wasn't correctly recovered has returned as null bytes. How can I remove every line from the file that contains these bytes?

Example of corrupt data


and a ton more...

I know NULL is equal to \x00.

How can I remove every line containing corrupt data with sed rather than removing the bytes individually?

There are so many variations of bytes/corrupt data that I doubt I would be able to discover all of them with regex..


To remove lines that contain byte 0 or bytes 128 to 255, you can use:

perl -ne 'print unless /[\0\200-\377]/'

Or with GNU grep built with PCRE support:

LC_ALL=C grep -vaP '[\0\200-\377]'

See also the strings command to extract what looks like printable text from data.

To just remove those bytes:

tr -d '\0\200-\377'
| improve this answer | |

You can try with this Perl command:

<in_file perl -lne's/\0//g; print if $_'

in_file is the input. Pipe redirection can be used.

It removes NULs globally. You can adjust the regular expression to suit your needs.

| improve this answer | |

You can remove all lines of yourfile that contain a null byte with gnu sed by

sed '/\x0/d' yourfile

This also works in a pipe:

cat yourfile | sed '/\x0/d'
| improve this answer | |

You might be able to use strings with a minimum length of, say, five characters

strings -w -n5 corrupted.txt
| improve this answer | |

Yes. You can do it like this:
sed -e '/\x00/d' [filename] > [new_filename]
If you want to edit the file in-place:
sed -i '/\x00/d' [filename]
You can also, combine the two, change the original file and keep a backup copy:
sed -i~ '/\x00/d' [filename]
That will delete any line of the file that contains at least 1 NULL.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.