I have a large binary file (nearly binary file !) which has many \r\n in it [ gets Generated in windows env and doesn't have controle over generating application ]. I want to remove \r from this big file [ =~ 1Gb ] to do further processing in another legacy app that shouts at \r's

Is there any easy way to do it the unix way?

  • Are you sure it's a binary file? If you open it in a text editor does it have \r\ns visible? It's easy to process a text file and change those up, but if you are really looking at a binary application that is quite different. What "further processing" do you have in mind? If this is a compiled binary you can't just muck around in the binary stuff, if it's not actually binary then the question is just how to change line feed types in a text file. – Caleb Jun 30 '11 at 10:34
  • it's complicated, yes is binary file , and yes it does have \r\n file has proprietary structure, binary file with headers separated by \r\n – Prashant Bhate Jun 30 '11 at 10:56
  • In that case the solutions for text files should work, although you might want to be more careful. The perl solution I mentioned only removes the carriage returns before newlines, so that might be the best option. You could also script perl to run on an in-place file instead of producing a new one if it's really 1G, but this way is saver. – Caleb Jun 30 '11 at 10:59
  • 1
    @Caleb sponge, perl -ip and friends still write a new file then replace the original below the hood. See Is there a way to modify a file in-place? for how to truly act on the existing file directly. – Gilles Jun 30 '11 at 20:52

Either it's binary or it isn't. In the event that your thing really isn't binary but just complicated text, here are a couple solutions.

First, you many Linux distros come with a utility called dos2unix that you can run on a file to convert the line ending style:

$ dos2unix original_file converted_file

You could also do this with tr like so:

$ tr -d '\r' < orignal_file > converted_file

Or with perl:

$ perl -pne 's/\r$//g' < orignal_file > converted_file
  • Liked perl syntax, how to replace it in file, instead of generating new one ? – Prashant Bhate Jun 30 '11 at 11:06
  • The -i argument makes it handle files in place. You can supply an argument to make it make a backup. perl -p -i -e 's/\r$//g' filename for no backup, or perl -p -i.bak -e 's/\r$//g' filename for with a backup. – Caleb Jun 30 '11 at 11:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.