I have a text file that has blocks of consecutive non empty lines separated by a blank line. I want to join the lines in a block (similar to the "J" key in the vi editor). Here I found the following script by "cfajohnson", that does the job:

awk 'BEGIN { RS = ""; OFS = " "}
           {$1 = $1; print }'

so processing the file




hello world
this is another line

(On two Solaris machines (SunOS 5.11 11.1 and SunOS 5.10 Generic_147440-16) I get a segmentation fault when the blocks were separated by three or more empty lines. On Linux it works also if they are separated by two or more lines)

From the manual of awk (Linux):

 Assigning a value to an existing field causes 
 the whole record to be rebuilt when $0 is referenced.
OFS         The output field separator, a space by default.
RS          The input record separator, by default a newline.

It seems that the script also works if one omits the

RS=" "

statement in the BEGIN-block (blank is the default value of RS) I don't understand why this script joins the lines, remove leading and trailing whitspace.

Can anybody explain how this script works?

  • 1
    On Solaris, try using /usr/xpg4/bin/awk instead of /usr/bin/awk – glenn jackman Jun 16 '16 at 19:16
  • @glennjackman: I tried xpg4-aswk on SunOS 5.10 Generic_147440-16 and there was no core dump. – miracle173 Jun 21 '16 at 8:37

You may omit the OFS=" " I think, but the RS="" (or equivalent) is essential in order to put awk into paragraph mode.

From the GNU awk manual, 4.8 Multiple-Line Records (other awks behave similarly, AFAIK):

Another technique is to have blank lines separate records. By a special dispensation, an empty string as the value of RS indicates that records are separated by one or more blank lines. When RS is set to the empty string, each record always ends at the first blank line encountered. The next record doesn’t start until the first nonblank line that follows. No matter how many blank lines appear in a row, they all act as one record separator. (Blank lines must be completely empty; lines that contain only whitespace do not count.)

In this mode, fields are still separated on whitespace by default, but whitespace now includes (single) newlines. The default output field separator is a single space, so all that's required to turn each multi-line record into a single line of space-separated fields is to force awk to rebuild the record variable $0, which is achieved as a side-effect of the assignment $1=$1 - see for example Famous Awk One-Liners Explained, Part II: Text Conversion and Substitution, 27. Delete both leading and trailing whitespaces from each line (trim).

  • thank you. 'man gawk' does not contain this information but I found it now in 'info gawk' – miracle173 Jun 21 '16 at 8:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.