In a text file, ignoring any trailing whitespace at the end of each line, I assume that if a line is not ended with a digit, then there is a line break between the line and the next line, and I would like to find these line breaks and then concatenate them into one line. For example

line 1
li
ne 2

There is a line break between the second and the third lines and I should modify the file to be

line 1
line 2

To find such line breaks, I need to do multiline matching. I does it by changing record separator, but the following doesn't work:

$ awk 'BEGIN{RS="";}; { if (match($0, /[^[:digit:] ] *\n/)) print $0;} ' inputfile

To concatenate two lines separated by a line break, I am still wondering.

Thanks.

  • setting RS to the empty string will turn on paragraph mode (records will be separated by runs of empty lines), not 'multiline matching' which is always on in awk. It's no wonder your script doesn't work, because it will just treat the whole file as a single record and print it, terminated by an extra newline (ORS). Also, there's absolutely no point in using the match() function, if you're not using its return value or the RSTART or RLENGTH variables. – mosvy Nov 13 at 17:02
up vote 1 down vote accepted

You could run something along the lines of

awk 'BEGIN{RS=SUBSEP; ORS="" } {print gensub(/([^0-9])\n/,"\\1","g",$0)}' ex
  • RS=SUBSEP sets the Register Separator to a value that is never present in a text file (slurps the input file to $0)
  • then do you favorite multiline transformation
  • Thanks. Do you know matching without substitution for multiline case? – Tim Nov 13 at 21:29
  • I was wondering if this reply doesn't work well sometimes? Why is this reply downvoted? – Tim Nov 13 at 22:30
  • Is RS="\f" also a working solution? – Tim Nov 13 at 22:40
  • 1
    This seems to add an empty line at the end of the output. I'm not sure exactly why at the moment. – Kusalananda Nov 13 at 22:40
  • 1
    @JJoao In general, print non-record data with printf and records with print. Since you're operating in "slurp mode" here (so to speak) and therefore do not really operate on records, it would be appropriate to use printf. – Kusalananda Nov 14 at 9:36

I would address it differently: by looping over the input until you find a "line-ending condition":

awk '{ 
       line=$0; 
       while($0 !~ /[[:digit:]] *$/ && getline > 0) { 
         line=line$0; 
       }
       print line
     }' < input

On an extended input file of:

line 1
li
ne 2
li
ne 
number 3
line 4

Or, more verbosely (to see the trailing space):

$ cat -e input
line 1$
li$
ne 2$
li$
ne $
number 3$
line 4$

The output is:

line 1
line 2
line number 3
line 4
  • Thanks. The script in your reply is very specific to the problem. I would like to see if there is a more general script, which can allow me to specify a multiline pattern and match (and substitute) the matches. – Tim Nov 13 at 16:50
  • What "multilne patterns" are you thinking of? – RudiC Nov 13 at 17:26
$ cat file
line 1
li
ne 2
lo
ng li
ne 3
$ awk 'line ~ /[0-9]$/ { print line; line = "" } { line = line $0 } END { print line }' file
line 1
line 2
long line 3

This accumulates an "output line" in the variable line, and whenever this variable ends with a digit, it is printed and reset. It is also printed at the very end to output the last line (whether complete or not).

Approximate sed equivalent (but with an explicit loop):

$ sed -e ':again' -e '/[0-9]$/{ p; d; }; N; s/\n//' -e 'tagain' file
line 1
line 2
long line 3

Small GNU sed?

sed ':L; /[0-9] *$/!{N; bL;}; s/\n//g' file

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.