I have an input file which contains both Unix (LF) and Windows (CR/LF) style newlines. (Specifically, it's XML from a Linux system, but it contains some raw HTTP headers, and HTTP prefers CRLF for headers):

    <response_page cause="default">
      <response_header>HTTP/1.1 200 OK^M
Cache-Control: no-cache^M
Pragma: no-cache^M
Connection: close</response_header>

I'm working on a gawk script to go through this file to make some simple tweaks to the XML* and the only problem is that it reads both LF and CRLF valid RS but only outputs LF regardless of what was there... In essence, it strips the CRs.

I've tried various things, the most ambitious being regex matching for RS and printing RT:

BEGIN { RS = "\r\n|\n"; go = "no" }
(go ~ /yes/) { 
    sub(/false/, "true", $0)
    go = "no"
($0 ~ /<signature signature_id="200000017">/) { 
    print "Found signature!"
    go = "yes"
    printf $0 RT

I would greatly appreciate any pointers on getting gawk to reproduce mixed-platform RS terminators.

* In this case, the simple tweak is to change 'false' to 'true' on the line following the line with the correct signature ID. I fully realize that using an XML parser would be the correct way to do this, but for such a lightweight need am trying to avoid buying into the howl of pain and angst that is XML parsing.


As it turns out, this solution works - when run under Linux. When run under Cygwin gawk, on Windows, the CRLF/LF distinction is apparently muted, and it does not work as expected. I am awarding the answer points to Peter.O, even though he essentially reiterated what I was trying, because he did so in a thorough manner that made me question my assumptions when I realized we were doing the same thing and mine didn't work.


You can use the built-in variable RT

RT is set each time a record is read. It contains the input text that matched the text denoted by RS, the record separator. This variable is a gawk extension.

printf '%s\n' LF CRLF$'\r' | 
  gawk 'BEGIN { RS = "\r\n|\n" }
       { printf($0 RT) }'

Output when piped to sed -n l - which shows CR as \r, and end-of-line as $ - which, to sed means that the next character is \n (or end-of-input.


However, if you want to toggle the terminator from CRLF to LF or vice-versa, the two actions are:

printf '%s\n' was-LF was-CRLF$'\r' | 
  gawk 'BEGIN { RS = "\r\n|\n" }
        RT == "\r\n" { printf($0 "\n") }
        RT == "\n"   { printf($0 "\r\n") }'

Output when piped to sed -n l


Note: You will need to use if for the tests when they aren't the first lines of (main section) code:

  gawk 'BEGIN { RS = "\r\n|\n" }
        { # some processing code here (before the tests)
          if( RT == "\r\n" ) { printf($0 "\n") }
          if( RT == "\n")    { printf($0 "\r\n") } }'
  • Thank you @peter-o - in fact, that's exactly what I was trying, but mine was failing due to Linux vs. Cygwin gawk behavior. Which I wouldn't have followed upon if your post wasn't thorough enough to convince me I needed to find what was missing. – gowenfawr Jun 9 '15 at 2:15

A straightforward solution is to treat just LF as a line ending, yank out the final CR if any, and print it out.

{ CR = (sub(/\r$/,"") ? "\r" : "") }
… { … print "stuff" CR }

The output will always end with a LF, even if the last input line isn't terminated.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.