multiline editing using perl or any other tool

Question

I'm editing my .uncrustify.cfg file to make it more readable. I just want to reformat like this:

Two lines in:

# Add or remove between the parens in the function type: 'void (*x)(...)'
sp_after_tparen_close      = ignore   # ignore/add/remove/force

One line out:

sp_after_tparen_close      = ignore   # ignore/add/remove/force#   Add or remove between the parens in the function type: 'void (*x)(...)'

Perl seems to be the way to go, but the syntax overwhelms me. When I have a spare decade, I'll learn it ;-)

To make it a bit more generic:

Two lines in:

#a comment line
some code

One line out:

some code # a comment line

==========================

John: Two lines done by hand:

nl_while_brace             = ignore   # I,A,R,F     # Add or remove newline between 'while' and '{'
nl_scope_brace             = ignore   # I,A,R,F     # Add or remove newline between 'scope (x)' and '{' (D)

... two pairs that didn't combine using your awk:

# Add or remove newline between 'unittest' and '{' (D)
nl_unittest_brace          = ignore   # I,A,R,F

# Add or remove newline between 'version (x)' and '{' (D)
nl_version_brace           = ignore   # I,A,R,F

Answer 1

sed '/^#/N;s/\(.*\)\n\([^#].*\)/\2 \1/;P;D'

That will handle your simple example in the question - any commented line that is followed by a line that is not a comment and consists of at least one character will be appended to the line that follows it.

So, running your example through it and the output is:

sp_after_tparen_close      = ignore   # ignore/add/remove/force # Add or remove between the parens in the function type: 'void (*x)(...)'

Running @John1024's example through it and the output is:

#
# Some comments
#

sp_after_tparen_close      = ignore   # ignore/add/remove/force # Add or remove between the parens in the function type: 'void (*x)(...)'

some code #a comment line
more code

# comment one
# comment two
still more code # comment three

To handle conditions like these, sed needn't loop. In this case the only lines that can possibly contain a \newline character are those that begin with a #hash because those are the only lines to which sed will add one.

When sed encounters a line beginning with a #hash it pulls in the Next input line and appends it to pattern space. sed then attempts to s///ubstitute:

• \(.*\) - as much as possible referenced as \1 immediately followed by...
• \n - a newline character immediately followed by...
• \([^#].*\) - at least one character that is not a #hash and everything remaining in pattern space...
• with \2 \1.

sed then Prints pattern space up to the first occurring \newline character and Deletes same before starting over to try again with what remains (if any).

Answer 2

I believe that this does what you want:

awk '/^[[:space:]]*[^#]/ && last ~ /^#/ {printf "%s %s",$0,last; last="";next} {print last; last=$0} END{print last}' sample.cfg

As an example, suppose that we have this input file:

#
# Some comments
#

# Add or remove between the parens in the function type: 'void (*x)(...)'
sp_after_tparen_close      = ignore   # ignore/add/remove/force

#a comment line
some code
more code

# comment one
# comment two
# comment three
still more code

The output is:

$ awk '/^[[:space:]]*[^#]/ && last ~ /^#/ {printf "%s %s",$0,last; last="";next} {print last; last=$0} END{print last}' uncrustify.cfg

#
# Some comments
#

sp_after_tparen_close      = ignore   # ignore/add/remove/force # Add or remove between the parens in the function type: 'void (*x)(...)'

some code #a comment line
more code

# comment one
# comment two
still more code # comment three

How it works

awk implicitly loops over every line in a file.

This script uses a single variable, last, which holds the previous line. In brief, as we go through each line, if the last line is a comment and the current line isn't, then we print out both lines together. Otherwise, we print out the last line.

• /^[[:space:]]*[^#]/ && last ~ /^#/ {printf "%s %s",$0,last; last="";next}

  If (a) this line is not a comment, and (b) the previous (last) line is a comment, then combine the last and current line and print them out. After that, last to empty. Then, skip the rest of the commands and jump to start over on the next line.

• {print last; last=$0}

  Otherwise, print the last line. Update last with the contents of the current line.

• END{print last}

  After we are done looping through all the lines in the file, print the contents of the last line.

Another example

Consider this input:

# Add or remove newline between 'unittest' and '{' (D)
nl_unittest_brace          = ignore   # I,A,R,F

# Add or remove newline between 'version (x)' and '{' (D)
nl_version_brace           = ignore   # I,A,R,F

The output is:

$ awk '/^[[:space:]]*[^#]/ && last ~ /^#/ {printf "%s %s",$0,last; last="";next} {print last; last=$0} END{print last}' new

nl_unittest_brace          = ignore   # I,A,R,F # Add or remove newline between 'unittest' and '{' (D)

nl_version_brace           = ignore   # I,A,R,F # Add or remove newline between 'version (x)' and '{' (D)

Answer 3

As per my comment, it is easiest to develop something like this in a short script rather than fiddling with command-one-liners. Also, you can keep it.

#!/usr/bin/perl
use strict;
use warnings FATAL => qw(all);

my $buffer = "";
my $linesBuffered = 0;

while (<STDIN>) {
# Check if this line is a just a comment.
    if ($_ =~ /^\s*#/) {
    # Assume multi-line comments should not be appended.
        if ($buffer) { $buffer .= $_ }
        else { $buffer = $_ }
        $linesBuffered++;
    } else {
        if ($linesBuffered > 1) {
        # Print multi-line comment.
            print "$buffer$_";
        # Reset buffer.
            $buffer = "";
            $linesBuffered = 0;
        } elsif ($buffer) {
        # Print buffered line with comment trailing.
            chomp $_;
            print "$_ $buffer";
        # Reset buffer.
            $buffer = "";
            $linesBuffered = 0;
        } else { print $_ }
    }
}

print $buffer if ($buffer);  

This could be used, e.g., ./filter.pl < .uncrustify.cfg > .uncrustify.copy. Notice that is not done in place so you would have to cp .uncrustify.copy .uncrustify.cfg afterward if you are happy. Since it reads from standard input, you can test it:

> ./filter.pl
what  <- stdin
what  <- stdout

Here it spit out the line immediately since it is not a comment. I won't bother indicating stdin and stdout in the next examples.

#okay
then
then #okay

In this case it buffered the comment line and appended it to the next (non-comment) line.

#foo
#bar
#foo
#bar

In this case it spit out a multi-line comment.

Couple of points:

• In perl an empty string ("") tests false.
• /^\s*#/ will match a line beginning with zero or more whitespace then a #.
• Unless chomped, input lines will have a newline at the end.

---

Here's the output from John1024's example:

#
# Some comments
#

sp_after_tparen_close      = ignore   # ignore/add/remove/force # Add or remove between the parens in the function type: 'void (*x)(...)'

some code #a comment line
more code

# comment one
# comment two
# comment three
still more code

Notice where # comment three is here in relation to John1024 and mikeserv's output. This is intentional, but the value of that depends on your requirements -- here my preference would be to assume that the last line of a multi-line comment should not be prepended to the next line of code, i.e., only single line comments followed by code are moved. Accomplishing this added complexity to the script; the $linesBuffered variable and logic relating to it would not be necessary otherwise.