How to integrate a multiline awk script in a shell script

Question

My question is a continuation of

How to parse a file to extract 3 digits numbers kept in a "group number"

I am trying to integrate in a single shell script a series of commands that

1. parse a european standard to extract a test sequence

2. convert the text encodings to utf8

3. process the result with the the awk routine that was provided to me on the post above.

4. save the content in a destination file

I have tentatively written the script below. I am able to achieve only step 1 and step 4, but neither step 2 nor step 3. I wonder if intermediate (temporary) file(s) should be created. I have tried to store the output of intermediate steps into variables, but without success. Any help also would be helpul regarding possible mistakes and the best way to do this.

#!/bin/bash
# creating the Latex code for a test procedure

awkcommand= "/usr/bin/awk
 '
    $1 == "Group" {printf("\\section{%s %d}\n", $1, $2); next}
    {
      title = sep = ""
      for (i=1; i<=NF; i++) 
        if ($i ~ /^[0-9][0-9][0-9]$/) {
          printf("\\subsection{%s} \n\\TestDetails{%d}\n", title, $i)
          break
        }
        else {
          title = title sep $i
          sep = FS
        }
    }
' 
"

sourcefolder="/Users/yves/Desktop/Test-folder-parsing/"
sourcefile="NFEN3545-001.pdf"
destfile="Latex-code.tex"
destfolder=$sourcefolder
destinationfilepath=${destfolder}${destfile}
extractioncmd="/usr/local/bin/pdftotext -layout -f 54 -l 54"
modifier=" -"
#textencodingcmd="/usr/bin/iconv -f L1 -t UTF-8" # Needed but not used

${extractioncmd}  ${sourcefolder}${sourcefile} ${modifier}  >  $destinationfilepath
exit 0

Answer 1

You can store the code passed to /usr/bin/awk in a variable and /usr/bin/awk in a separate variable like so (untested):

awk=/usr/bin/awk

awkcommand='
$1 == "Group" {printf("\section{%s %d}\n", $1, $2); next}
{
title = sep = ""
for (i=1; i<=NF; i++) 
  if ($i ~ /^[0-9][0-9][0-9]$/) {
    printf("\subsection{%s} \n\TestDetails{%d}\n", title, $i)
    break
  }
  else {
    title = title sep $i
    sep = FS
  }
}
'

Usage:

$awk "$awkcommand"

Note that I changed the double quotation marks to single quotation marks. Within double quotation marks, $i is substituted by the contents of the shell variable i. Within single quotation marks, it is a literal $i, which is what awk expects to see.

Also, you weren't escaping the double quotation marks within the string so awk never saw

$1 == "Group" {printf("\section{%s %d}\n", $1, $2); next}

Instead, it saw

<contents of shell $1> == Group {printf(\section{%s %d}\n, <contents of shell $1>, <contents of shell $2>); next}

If $1 and $2 were empty, awk saw

 == Group {printf(\section{%s %d}\n, , ); next}

Are you sure storing the command location is necessary? You can usually depend on finding awk within a directory in your user's path. If you don't use the full path to awk, there is no reason to parameterize awk.