Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Is it possible to use the find command to find all the "non-binary" files in a directory? Here's the problem I'm trying to solve.

I've received an archive of files from a windows user. This archive contains source code and image files. Our build system doesn't play nice with files that have windows line endings. I have a command line program (flip -u) that will flip line endings between *nix and windows. So, I'd like to do something like this

find . -type f | xargs flip -u

However, if this command is run against an image file, or other binary media file, it will corrupt the file. I realize I could build a list of file extensions and filter with that, but I'd rather have something that's not reliant on me keeping that list up to date.

So, is there a way to find all the non-binary files in a directory tree? Or is there an alternate solution I should consider?

share|improve this question
You could use the file utility somewhere in your script/pipeline to identify whether the file is data or text – lk- Aug 24 '12 at 18:59
up vote 7 down vote accepted

I'd use file and pipe the output into grep to find text files, then extract just the filename portion of file's output and pipe that into xargs.

something like:

file * | grep 'ASCII text' | awk -F: '{print $1}' | xargs -d'\n' -r flip -u

Note that the grep searches for 'ASCII text' rather than any just 'text' - you probably don't want to mess with Rich Text documents or unicode text files etc.

You can also use find (or whatever) to generate a list of files to examine with file:

find /path/to/files -type f -print0 | xargs -0r file | grep 'ASCII text' | \
awk -F: '{print $1}' | xargs -d'\n' -r flip -u

The -d'\n' argument to xargs makes xargs treat each input line as a separate argument, thus catering for filenames with spaces and other problematic characters. i.e. it's an alternative to xargs -0 when the input source doesn't or can't generate NULL-separated output (such as find's -print0 option). According to the changelog, xargs got the -d/--delimiter option in Sep 2005 so should be in any non-ancient linux distro (I wasn't sure, which is why I checked - I just vaguely remembered it was a "recent" addition).

Note that file is not perfect. It's very good at detecting the type of data in a file but can occasionally get confused.

I have used numerous variations of this method many times in the past with success.

share|improve this answer
Thanks for this solution! For some reason file displays English text rather than ASCII text on my Solaris system, so I modified that portion accordingly. Also, I replaced awk -F: '{print $1}' with the equivalent cut -f1 -d:. – Andrew Cheong Dec 10 '13 at 18:12

No. There is nothing special about a binary or non-binary file. You can use heuristics like 'contains only characters in 0x01–0x7F', but that'll call text files with non-ASCII characters binary files, and unlucky binary files text files.

Now, once you've ignored that...

zip files

If its coming from your Windows user as a zip file, the zip format supports marking files as either binary or text in the archive itself. You can use unzip's -a option to pay attention to this and convert. Of course, see the first paragraph for why this may not be a good idea (the zip program may have guessed wrong when it made the archive).

zipinfo will tell you which files are binary (b) or text (t) in its zipfile listing.

other files

The file command will look at a file and try to identify it. In particular, you'll probably find its -i (output MIME type) option useful; only convert files with type text/*

share|improve this answer

A general solution to only process non-binary files in bash using file -b --mime-encoding:

while IFS= read -d '' -r file; do
  [[ "$(file -b --mime-encoding "$file")" = binary ]] &&
    { echo "Skipping   $file."; continue; }

  echo "Processing $file."

  # ...

done < <(find . -type f -print0)

And if you want to call one command with all these files as parameters:

find . -type f -exec sh -c '[ "$(file -b --mime-encoding "$0")" != binary ]' {} \; -exec MY_COMMAND {} \+

(The return code of the first -exec call determines whether the second one happens. Replace \+ with \; to call it multiple times one-by-one instead.)


find . -type f -exec sh -c '[ "$(file -b --mime-encoding "$0")" != binary ]' {} \; -print0 | xargs -0 MY_COMMAND

Both will result in MY_COMMAND ascii_file1 ascii_file2 ascii_file3 ... being called.


Good news everyone! I contacted the author of the file utility and he added a nifty -00 paramter in version 5.26 (released 2016-04-16, is e.g. in Arch already) which prints file\0result\0 for multiple files fed to it at once, this way you can do e.g.:

find . -type f -exec file -00 --mime-encoding {} + | awk 'BEGIN{ORS=RS="\0"}{if(NR%2){f=$0}else if($0!="binary")print f}' | …

(The awk part is to filter out every file that isn't non-binary. ORS is the output seperator.)

No more multiple calls to file and sh necessary! The previous method took nearly seven times as long in a test I did.

Can be also used in a loop of course:

while IFS= read -d '' -r file; do

  echo "Processing $file."

  # ...

done < <(find . -type f -exec file -00 --mime-encoding {} + | awk 'BEGIN{ORS=RS="\0"}{if(NR%2){f=$0}else if($0!="binary")print f}')


I created a little bash script for filtering out binary files which can work with the newer and older versions of file (in the newer versions it uses the much faster method):


# Expects files as arguments and returns the ones that do
# not appear to be binary files as a zero-seperated list.
#   filter_binary_files.sh [FILES...]
#   find . -type f -mtime +5 -exec ./filter_binary_files.sh {} + | xargs -0 ...

[[ $# -eq 0 ]] && exit

if [[ "$(file -v)" =~ file-([1-9][0-9]|[6-9]|5\.([3-9][0-9]|2[6-9])) ]]; then
  file -00 --mime-encoding -- "$@" |
    awk 'BEGIN{ORS=RS="\0"}{if(NR%2){f=$0}else if($0!="binary")print f}'
  find "$@" -exec sh -c \
    '[ "$(file -b --mime-encoding -- "$0")" != binary ]' {} \; -print0
share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.