Join Lines using Vim Editor

Question

I have a very long text file with the data repeated like this

510068893                                     
510069716                                     
620087756                                     
510068729                                     
620089710                                     
560034147                                     
510069875

M                                                      
M                                                   
M
M                                                      
M                                                   
M                                                
M

1957
5/2/1980
1960
1978
5/8/1988
1984
1981

ABASSI FADHIL KHAMIS
ABDALLA HAMAD JUMA
ABDALLA ALI MAHADHI
ABDALLA ALI MFAKI
ABDALLA HAMAD KHAMIS
ABDALLA JUMA ALI
ABDALLA JUMA MABROUK

Sokoni                                                 
Sokoni                                                
Sokoni                                                
Sokoni                                                
Sokoni                                                
Sokoni                                                
Sokoni

510068695                                     
510068680                                     
620086828                                     
560034266                                     
620088288                                     
560033477                                     
510068873

M                                                      
M                                                   
M
M                                                      
M                                                   
M                                                
M

5/10/1983
1985
1937
1951
1974
14/5/1984
1987

ABDALLA KHAMIS ALI
ABDALLA KHAMIS HAMAD
ABDALLA KHAMIS USHURU
ABDALLA MOHAMMED JUMA
ABDALLA MOHAMMED SALEH
ABDALLA OMAR ABEID
ABDALLA OMAR NUHU

Sokoni                                                 
Sokoni                                                
Sokoni                                                
Sokoni                                                
Sokoni                                                
Sokoni                                                
Sokoni

I want to join every first line of each group to form one line and repeat for every second line of each group until I finish. How can I do that using Vim or Perl Script.

The output should look like

510068893, M, 1957, ABASSI FADHIL KHAMIS, Sokoni 510069716, M, 5/2/1980, ABDALLA HAMAD JUMA, Sokoni
620087756, M, 1960, ABDALLA ALI MAHADHI, Sokoni 510068729, M, 1978, ABDALLA ALI MFAKI, Sokoni
etc.

Answer 1

In case there's always data to collect for 7 entries (plus a blank line) then this awkprogram will collect the respective data and print them at the end:

awk '{a[NR%8]=a[NR%8]" "$0} END{for (i=1;i<=7;i++) print a[i]}'

(Note that your data samples seem to have a lot of trailling blanks; if you want those blanks removed instead of joined, please adjust your question appropriately, then I'll adjust my answer accordingly.)

Edit: To create the comma delimiters, as in your sample output in your above comment, use:

awk '{a[NR%8]=a[NR%8]", "$0} END{for (i=1;i<=7;i++) print substr(a[i],3)}'

Answer 2

gawk ' BEGIN{RS=""; FS="\n"}
            {for(i=1;i<NF;i++) {a[i]=a[i]" || "$i} } 
       END  {for(i in a){ print a[i]}}'

(this solution may change the order of the lines; next version keeps line order).

Even better, improve it and create a reusable command "vert_transpose"

#!/usr/bin/gawk -f

BEGIN {RS=""; FS="\n"}
l<=NF {l=NF}
      {for(i=1;i<=NF;i++) { a[i]=a[i]" || "$i } } 
END   {for(i=1;i<=l ;i++) { print a[i]        } }

chmod and install it in some $HOME/bin; and whenever necessary use it as common command:

• in bash: vert_transpose longfile
• inside vim: :%! vert_transpose

Answer 3

To remove trailing spaces from each line and add empty line at the end:

sed 's/\s*$//;$a\\' very.long.text.file

which can be piped through

awk '
BEGIN{
 RS="\n\n"
 FS="\n"}
{for (i=1;i<=NF;i++)
     A[i]=A[i] s $i
 s=","}
NR%5==0{
 for(i in A)
     printf "%s,%g\n" ,A[i]
 s=""
 delete A
 print s}'

which empties memory by each 40th line to be suitable for large files.

Answer 4

Use Ctrl+V to enable the visual block and select the lines to be joined, then hit Shift+J (i.e., capital J) to merge the lines into a single line.