How change first rows in column format of next's rows, replacing * by a number and perform operations between columns?

Question

I have the following file which has the first two rows as if they were a single-column (i.e. a continuous string). I would like to separate them into columns and replace the "*" character with an "x" number giving the scientific-notation and column-format as fourth and fifth rows.

0.001000000*********************************************
0.061059059-3524.927327218-3524.938421865***************
0.121118118 -887.564833130 -887.569649256-6250.350946527
0.181177177 -387.169559377 -387.173137963-2743.981985633
0.241236236 -223.812193853 -223.815321341-1504.799155086
0.301295295 -134.073058536 -134.075910507 -924.916305653
0.361354354  -76.668692929  -76.671412688 -612.480371134

Note that lines 1,2 & 3 have the same problem of having 2 consecutive columns as if there were a single-column (I would like to put a space between them). I would also like to perform a complex operation such as sqrt((sqrt($2 ^ 2 + $4 ^ 2) + $2) / 2) between columns.

Expected Results:

0.001000000 -3524.927327218 -3524.938421865 -6250.350946527
0.061059059 -3524.927327218 -3524.938421865 -6250.350946527
0.121118118  -887.564833130  -887.569649256 -6250.350946527
0.181177177  -387.169559377  -387.173137963 -2743.981985633
0.241236236  -223.812193853  -223.815321341 -1504.799155086
0.301295295  -134.073058536  -134.075910507  -924.916305653
0.361354354   -76.668692929   -76.671412688  -612.480371134

I would like to know if there is any solution to my problem.

Answer 1

Seems to me that you have two very different problems:

• Incomplete lines that you need to fill in
• Numbers that are appended with no separator

While this is probably doable in one awk invocation, I would 1 invocation for each task for the sake of simplicity.

From your sample input/output, I am assuming that your numbers always have 9 decimals.

Dealing with the "no separator" issue

awk '{
         while ($0 ~ /[0-9]+\.[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][^ ]/) {
             $0=gensub(/([0-9]+\.[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9])([^ ])/, "\\1 \\2", $0)
        }
        print
      }' input.txt > first_step.txt

Note: if your gawk version is >= 4.0, you can replace [0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9] with [0-9]{9}, which yields:

awk '{
         while ($0 ~ /[0-9]+\.[0-9]{9}[^ ]/) {
             $0=gensub(/([0-9]+\.[0-9]{9})([^ ])/, "\\1 \\2", $0)
        }
        print
      }' input.txt > first_step.txt

(Kind of easier to read right ?) With this we have first_step.txt looking like this:

0.001000000 *********************************************
0.061059059 -3524.927327218 -3524.938421865 ***************
0.121118118 -887.564833130 -887.569649256 -6250.350946527
0.181177177 -387.169559377 -387.173137963 -2743.981985633
0.241236236 -223.812193853 -223.815321341 -1504.799155086
0.301295295 -134.073058536 -134.075910507 -924.916305653
0.361354354  -76.668692929  -76.671412688 -612.480371134

Replacing '*' with values from following lines

This is a bit tricky to achieve too. Assuming those '*' lines occur only at the beginning of your file. We will first make every line have 4 fields:

awk '/\*/ {
              a=""
              for (i=1; i < 5; i++) {
                  if (i < NF) a=a" "$i
                  else a=a" ***************"
              }
              print a; next
           }
           {print}' first_step.txt > second_step.txt

Output in second_step.txt

 0.001000000 *************** *************** ***************
 0.061059059 -3524.927327218 -3524.938421865 ***************
0.121118118 -887.564833130 -887.569649256 -6250.350946527
0.181177177 -387.169559377 -387.173137963 -2743.981985633
0.241236236 -223.812193853 -223.815321341 -1504.799155086
0.301295295 -134.073058536 -134.075910507 -924.916305653
0.361354354  -76.668692929  -76.671412688 -612.480371134

Now, the fun part...

awk 'BEGIN{first_lines=0}
     /\*/ {for (i=1; i<NF+1;i++) a[NR, i]=$i; next}
     first_lines != 1 {for (i=1; i<NF+1;i++) {a[NR, i]=$i};
                       for (i=NR-1; i > 0; i--) {
                           for (j=1; j < NF +1; j++) {
                               if (a[i, j] ~ /^\**$/) a[i, j] = a[i+1, j]
                           }
                       }
                       for (i=1; i < NR+1; i++) {
                           for (j=1; j < NF +1; j++) {
                               printf("%16s", a[i, j])
                           }
                           printf("\n")
                       }
                       first_lines = 1
                       next
                      }
                      {for (i=1;i<NF+1; i++) printf("%16s", $i)
                       printf("\n")
                      }' second_step.txt > output.txt

Output:

     0.001000000 -3524.927327218 -3524.938421865 -6250.350946527
     0.061059059 -3524.927327218 -3524.938421865 -6250.350946527
     0.121118118  -887.564833130  -887.569649256 -6250.350946527
     0.181177177  -387.169559377  -387.173137963 -2743.981985633
     0.241236236  -223.812193853  -223.815321341 -1504.799155086
     0.301295295  -134.073058536  -134.075910507  -924.916305653
     0.361354354   -76.668692929   -76.671412688  -612.480371134