Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I am a beginner, and trying to write a shell script to parse a text file that is extracted from a standardization pdf file. I would like for each test group (identified by Group 0, Group 1... to get the list of test numbers, such as 101, 102, 412... for group 0. I have tried sed, awk, but I do not have sufficient skills to succeed. Ideally I would like to get the output translated into LaTeX code, ie each output item surrounded by a suitable string, such as

\section{Group0}
\Testdetails{101}
\Testdetails{102}
...............
\section{Group1}
\Testdetails{305}
................

This is the source file.

                                                Table 6

                       Tests                     EN 2591-                   Remarks

                                                            All models
 Group 0
 Visual examination                                101
 Examination of dimensions and mass                102      To be performed on one pair per layout, in
                                                            sealed and un-sealed versions
 Contact insertion and extraction forces           412      To be performed on one pair per layout, in
                                                            sealed and un-sealed versions
 Measurement of insulation resistance              206      Only specimens of group 6
 Voltage proof test                                207      Only specimens of group 6
 Contact resistance - Low level                    201
 Contact resistance at rated current               202
 Mating and unmating forces                        408      On specimens of groups 2, 4 and 6
 Visual examination                                101
 Group 1
 Rapid change of temperature                       305
 Visual examination                                101
 Interfacial sealing                               324
 Measurement of insulation resistance              206      Immersed connectors
 Voltage proof test                                207      Immersed connectors
 Insert retention in housing (axial)               410
 Contact retention in insert                       409
 Mechanical strength of rear accessories           420
 Contact retention system effectiveness            426
 (removable contact walkout)
 Visual examination                                101
 Group 2
 Contact retention in insert                       409
 Rapid change of temperature                       305
share|improve this question

3 Answers 3

up vote 3 down vote accepted
awk '
    $1 == "Group" {printf("\\section{%s%d}\n", $1, $2); next}
    {for (i=1; i<=NF; i++) 
        if ($i ~ /^[0-9][0-9][0-9]$/) {
            printf("\\Testdetails{%d}\n", $i)
            break
        }
    }
' 

Update based on comment:

awk '
    $1 == "Group" {printf("\\section{%s %d}\n", $1, $2); next}
    {
      title = sep = ""
      for (i=1; i<=NF; i++) 
        if ($i ~ /^[0-9][0-9][0-9]$/) {
          printf("\\subsection{%s} \\Testdetails{%d}\n", title, $i)
          break
        }
        else {
          title = title sep $i
          sep = FS
        }
    }
' 
share|improve this answer
    
I generate the source file by "/usr/local/bin/pdftotext -layout -f 54 -l 60 <path-to-pdf-file> - | iconv -f L1 -t UTF-8" . How can I best link the two commands to create the LaTeX code in a single step if possible? –  Yves May 8 '12 at 15:46
    
@Yves, just extend your existing pipeline: pdftotext ... | iconv ... | awk ... –  glenn jackman May 8 '12 at 16:07
    
Btw, is there a way to maintain the space between"Group" and the group number in the target file, ie \Section{Group 0} and not \Section{Group0}. I feel I would now need also to add the test title in the target code, such as \subsection{Examination of dimensions and mass} \TestDetails{102} This would mean a way to use for the title a many necessary fields as needed, since printf("\\subsection{"$1" "$2"} \\TestDetails{%d}\n", $i) returns obviously a truncated label, such as \subsection{Examination of} \TestDetails{102} –  Yves May 10 '12 at 4:35
    
@Yves, see update. –  glenn jackman May 10 '12 at 12:44
    
Fantastic. The Latex code generated provides exactly the desired result, and will enable to automate the process almost entirelY...Many thanks –  Yves May 12 '12 at 10:50

One way with perl using regexp and assuming infile has the content you posted in the question.

Content of script.pl:

use warnings;
use strict;

while ( <> ) { 
    chomp;
    if ( m/\A\s*(Group)\s*(\d+)/ ) { 
        printf qq[\\Section{%s}\n], $1 . $2; 
        next;
    }   

    if ( m/\s(\d{3})(?:\s|$)/ ) { 
        printf qq[\\Testdetails{%s}\n], $1; 
    }   
}

Run it like:

perl script.pl infile

With following output:

\Section{Group0}                                      
\Testdetails{101}                                      
\Testdetails{102}                                      
\Testdetails{412}                                      
\Testdetails{206}                                      
\Testdetails{207}                                      
\Testdetails{201}                                      
\Testdetails{202}                                     
\Testdetails{408}                                      
\Testdetails{101}                                      
\Section{Group1}                                      
\Testdetails{305}                                     
\Testdetails{101}                                     
\Testdetails{324}                                     
\Testdetails{206}                                      
\Testdetails{207}                                        
\Testdetails{410}
\Testdetails{409}
\Testdetails{420}
\Testdetails{426}
\Testdetails{101}
\Section{Group2}
\Testdetails{409}
\Testdetails{305}
share|improve this answer
    
Thanks. I had been trying for a long time.... –  Yves May 8 '12 at 15:33
    
Thanks a lot. This is quite fascinating. I will need time to explore the code.. I have just realized that in some cases, the standards are bilingual, therefore my source file contains duplicated lines, ie contains two tables 6 such as above (one in english, one in a second language). Is there an easy way to alter the code to remove the duplicated groups and associated tests numbers? –  Yves May 8 '12 at 16:49
    
@Yves: Do you mean not to process repeated groups? Only first group-0, only first group-1, and so on? –  Birei May 8 '12 at 19:13
    
Yes. As a matter of fact, I am trying another approach, trying to get rid of the duplicated lines while extracting the text file from the original pdf document. I have noticed the pages in english are on even pages while the ones I want to discard are odd. I have not yet found a clean way to do this, since the routine I use can set a range of pages, (see below from 50 to 56) but not the even ones only:/usr/local/bin/pdftotext -layout -f 50 -l 56 <path to pdf file> - | iconv -f L1 -t UTF-8 | –  Yves May 9 '12 at 17:13

For completeness here is a sed version:

sed -n -e 's#^ *Group \([0-9]\+\).*#\\Section{Group\1}#p' \
       -e 's#.*\b\([0-9][0-9][0-9]\)\b.*#\\Testdetails{\1}#p'
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.