0

I have a file looks like below:

1 
4 5 6 7 19
20
22
24 26 27 
29
30
31
32 
34 
40 
50 
56 
58
100
234 235 270 500
1234 1235 1236 1237
2300
2303
2304
2307
2309

As it is clear there are some rows with more than 1 column and some others with only one single column. I'd like to join together single-column rows such that there are at most 4 columns on each combined row. So the output should look like this:

1  
4 5 6 7 19
20 22
24 26 27 
29 30 31 32
34 40 50 56 
58 100
234 235 270 500
1234 1235 1236 1237
2300 2303 2304 2307
2309

Any suggestions on how do this, considering that the real data is big?

4

A little bit idiomatic but working with gnu awk:

awk '{printf "%s",(NF==1?$0 FS:(c==0?"":RS) $0 RS)} \
{(NF==1?++c:c=0)} \
c==4{printf "\n";c=0} \
END{printf "\n"}' file

#Output
1 
4 5 6 7 19
20 22 
24 26 27
29 30 31 32 
34 40 50 56 
58 100 
234 235 270 500
1234 1235 1236 1237
2300 2303 2304 2307 
2309 

Explanation:
awk variables:
NF=Number of Fields
FS=Field Separator = space by default
RS=Record Separator= new line by default.
c=counter

Line1: {printf "%s",(NF==1?$0 FS:(c==0?"":RS) $0 RS)}: nested ternary if operations

#Single ternary if operation:
condition?true action:false action
#Nested if operations:  
condition1?true action 1:(condition2:true action2:false action2) #nested ternary if operations   
-------------------------[            ^ false action1 ^        ]   

This can be explained in pseudocode like:

if NF==1 then print $0 and print FS   
else (if c==0 then print "" else print RS) and print $0 and print RS again   

Line 2: {(NF==1?++c:c=0)} : Another ternary if operation that can be expressed as:

If NF==1 (line has one field) 
then increase counter c by one 
else reset counter c.  

Line 3 : c==4{printf "\n";c=0} Classic awk syntax : condition{action}

If counter c==4 then print a new line and reset counter c

Line 4: END{printf "\n"}' file : This justs prints a new line at the end of the script.

  • 2
    Ugh, this might be a bit more understandable with at least a bit of explanation added, or at least a more friendly formatting? Just saying. :) – ilkkachu Jun 1 '17 at 12:20
  • @ilkkachu OK - Added some basic explanation and a bit better formatting. Thanks for your comment. – George Vasiliou Jun 1 '17 at 13:57
  • Are extra interfield spaces and trailing spaces a big deal? Your solution have them both :) - I have run it just now. Should we care about it or it is unimportant in this case? I was care about it in the my solution. – MiniMax Jun 1 '17 at 14:50
  • 1
    @MiniMax Default field separator of awk (FS) is actually whitespace and this include one or more spaces. A basic awk like awk '{print $2}' <<<"one two" will work either with one space between fields or with more spaces . – George Vasiliou Jun 2 '17 at 7:37
2

You may use sed to get what you want:

sed -e '
   /./!b
   /[^[:space:]]/!b
   /[^[:space:]][[:blank:]]\{1,\}[^[:space:]]/b

   :loop
      $q;N
      /\n.*\S[[:blank:]]\+\S/b
      s/\n/ /;tdummy
      :dummy
      s/[[:space:]]\{1,\}/&/3;t
   bloop
' yourfile


Explanations

  • Skip empty, blank, and lines with NF > 1.
  • Setup a do-while loop at the point where the pattern space holds a single-field line.
  • We grab the next line and check whether it has NF > 1, at which point we print the whole pattern space and go back to reading the next line.
  • Now we know that the next line also is single-field, so we go ahead and clip the newline joining these two portions in the pattern space.
  • Does the pattern space have 3 spaces chunks yet? If yes then we print the whole of pattern space and start reading the next line.
  • Otherwise, we branch back to the loop which will in turn read the next line but attach it to the existing pattern space.

Result

1
4 5 6 7 19
20 22
24 26 27
29 30 31 32
34 40 50 56
58 100
234 235 270 500
1234 1235 1236 1237
2300 2303 2304 2307
2309
2

Usage: ./join_rows.awk input.txt

Check shebang #!/usr/bin/awk -f, because the awk location may differ on your system.

#!/usr/bin/awk -f

BEGIN {
    count = 1;
}

{
    if (NF == 1) {
        if (count > 1 && count <= 4) printf " ";

        printf "%s", $1;
        count++;

        if (count > 4) {
            printf "\n";
            count = 1;
        }
    } else {
        if (count > 1) printf "\n";

        print;
        count = 1;
    }
}

END {
    if(count > 1) printf "\n";
}

Output:

1
4 5 6 7 19
20 22 
24 26 27  
29 30 31 32
34 40 50 56
58 100
234 235 270 500
1234 1235 1236 1237 
2300 2303 2304 2307
2309
  • +1 for clarity. I would probably start count from zero, now I think count == 1 means that there weren't any single-column rows. – ilkkachu Jun 1 '17 at 12:19
1

Extended gawk approach:

rearrange_columns.awk script:

#!/bin/awk -f
function printRow(a, i, v)
{
    for (i in a) {
        printf "%s ", a[i]
    }
   print ""
   delete a
}
NF <= 2{
    for (i=1; i<=NF; i++) { 
        a[++c] = $i 
        if (length(a) == 4) {
            c = 0 
            printRow(a) 
        }
    }
}
NF > 2{
    if (length(a) > 0) {
        c = 0
        printRow(a)
    }
    print $0 
}
END{ print }

Usage:

awk -f rearrange_columns.awk yourfile

The output:

1 
4 5 6 7 19
20 22 
24 26 27 
29 30 31 32 
34 40 50 56 
58 100 
234 235 270 500
1234 1235 1236 1237
2300 2303 2304 2307 
2309
  • I added one row with two columns and your solution broke. For example, add one column to the 9-th row, new row will be 32 87. I got this string in the output: 29 30 31 32 87 34 40 50 56 58 100 – MiniMax Jun 1 '17 at 13:34
  • @MiniMax, be more precise. You added one row with two columns in what position? – RomanPerekhrest Jun 1 '17 at 13:36
  • I was writing the row number - "to the 9-th row". Row which have one column with 32 value. – MiniMax Jun 1 '17 at 13:41
  • @MiniMax, ok, fixed. Thanks for checking out – RomanPerekhrest Jun 1 '17 at 19:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.