Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Is it possible to use the find command to find all the "non-binary" files in a directory? Here's the problem I'm trying to solve.

I've getting an archive of files from a windows user. This archive contains source code and image files. Our build system doesn't play nice with files that have windows line endings. I have a command line program (flip -u) that will flip line endings between *nix and windows. So, I'd like to do something like this

find . -type f | xargs flip -u

However, if this command is run against an image file, or other binary media file, it will corrupt the file. I realize I could build a list of file extensions and filter with that, but I'd rather have something that's not reliant on me keeping that list up to date.

So, is there a way to find all the non-binary files in a directory tree? Or is there an alternate solution I should consider?

share|improve this question
    
You could use the file utility somewhere in your script/pipeline to identify whether the file is data or text –  lk- Aug 24 '12 at 18:59
add comment

2 Answers 2

up vote 5 down vote accepted

I'd use file and pipe the output into grep to find text files, then extract just the filename portion of file's output and pipe that into xargs.

something like:

file * | grep 'ASCII text' | awk -F: '{print $1}' | xargs -d'\n' -r flip -u

Note that the grep searches for 'ASCII text' rather than any just 'text' - you probably don't want to mess with Rich Text documents or unicode text files etc.

You can also use find (or whatever) to generate a list of files to examine with file:

find /path/to/files -type f -print0 | xargs -0r file | grep 'ASCII text' | \
awk -F: '{print $1}' | xargs -d'\n' -r flip -u

The -d'\n' argument to xargs makes xargs treat each input line as a separate argument, thus catering for filenames with spaces and other problematic characters. i.e. it's an alternative to xargs -0 when the input source doesn't or can't generate NULL-separated output (such as find's -print0 option). According to the changelog, xargs got the -d/--delimiter option in Sep 2005 so should be in any non-ancient linux distro (I wasn't sure, which is why I checked - I just vaguely remembered it was a "recent" addition).

Note that file is not perfect. It's very good at detecting the type of data in a file but can occasionally get confused.

I have used numerous variations of this method many times in the past with success.

share|improve this answer
    
Thanks for this solution! For some reason file displays English text rather than ASCII text on my Solaris system, so I modified that portion accordingly. Also, I replaced awk -F: '{print $1}' with the equivalent cut -f1 -d:. –  Andrew Cheong Dec 10 '13 at 18:12
add comment

No. There is nothing special about a binary or non-binary file. You can use heuristics like 'contains only characters in 0x01–0x7F', but that'll call text files with non-ASCII characters binary files, and unlucky binary files text files.

Now, once you've ignored that...

zip files

If its coming from your Windows user as a zip file, the zip format supports marking files as either binary or text in the archive itself. You can use unzip's -a option to pay attention to this and convert. Of course, see the first paragraph for why this may not be a good idea (the zip program may have guessed wrong when it made the archive).

zipinfo will tell you which files are binary (b) or text (t) in its zipfile listing.

other files

The file command will look at a file and try to identify it. In particular, you'll probably find its -i (output MIME type) option useful; only convert files with type text/*

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.