2

I have a file, 4.txt that contains full paths to *.cfg files as well as additional data I need to strip for the final report (5.csv).

For example /source/EDDG/env1/dom1/proj/config/test.cfg

<ListVariable name="selected_lookups">          
 <CompoundVariableValue>
  <StringVariableValue name="lookup_name" value="CUSTOMER_1"/>
  <StringVariableValue name="business_name" value="DEVCUSTOMER"/>
  <StringVariableValue name="sample_data_path"value="/dev/.dat"/>
 </CompoundVariableValue>
 <CompoundVariableValue>
   <StringVariableValue name="lookup_name" value="CODE_1"/>
   <StringVariableValue name="business_name"value="CONCUSTOMER"/>
 </CompoundVariableValue>
</ListVariable>

(AND THIS SEQUENCE REPEATS FOR ~238 TIMES WITH DIFFERENT DATA BETWEEN <ListVariable * > and </ListVariable>.

Now I need to get 4 values from this file piped into a csv file... e.g.

DOM,    PROJ, CFG,      LOOKUP NAME VALUE(s) (thr can be many per cfgfile) 
source, EDGE, test.cfg, CUSTOMER_1
                      , CONCUSTOMER  (second lookup name value)
... repeat for all cfg files in 4.txt

In order to acquire this data, I have the following loop and it works great for the first 3 columns but not for the fourth column.

for COL_VAL in `cat 4.txt | grep '/source/EDDG*'` ;
do
    DOM=`echo "${COL_VAL}" | awk -F'/' '{ print $7 }'`
    PROJ=`echo "${COL_VAL}" | awk -F'/' '{ print $8 }'`
    CGF=`echo "${COL_VAL}" | awk -F'/' '{ print $10 }'`
    LKP=`echo "${COL_VAL}" | grep 'name="lookup_name" value="' | awk -F'value="' '{ print $2 }' | awk -F'_1' '{ print $1 }'`

    echo "${DOM},${PROJ},${CFG},${LKP}"

   done < ${TMPDIR}/4.txt > ${TMPDIR}/5.csv

So, I tried something like this nested loop:

for COL_VAL in `cat 4.txt | grep '/source/EDDG*'` ;
do
    DOMN=`echo ${COL_VAL} | awk -F'/' '{ print $7 }'`
    PROJ=`echo ${COL_VAL} | awk -F'/' '{ print $8 }'`
    APFG=`echo ${COL_VAL} | awk -F'/' '{ print $10 }'`

    for LOOK_UP in `cat 4.txt | grep 'name="lookup_name" value="'` ;
    do
        ULKP=`echo "${LOOK_UP}" |  awk -F'value="' '{ print $2 }' | awk -F'_1' '{ print $1 }'`
    done

    echo "${DOMN},${PROJ},${APFG},${ULKP}"

done < ${TMPDIR}/4.txt > ${TMPDIR}/5.csv

This populates the 4th column but with the same data. And, strangely to me, the data that does go in the 4th col is the value of the absolute last lookup name in 4.txt which is "'name="lookup_name" value=XYZ'"

e.g.

DOM,    PROJ, CFG,      LOOKUP NAME VALUE(s) 
source, EDGE, test.cfg, XYZ
      ,     ,         , XYZ
... repeat for all cfg files in 4.txt
  • 1
    So your input is a mix between plain text and XML? It would have been nearly trivial if it was one single XML file. – Kusalananda Sep 7 '18 at 18:49
  • Yeah pretty much! Any ideas? – SSDdude Sep 7 '18 at 18:58
  • Please clarify whether the leading line numbers are actually part of the input file text. That is, does cut -f1 -d' ' 4.txt | xargs echo output 1 2 3? – agc Sep 9 '18 at 11:08
  • Sorry, the numbers ARE NOT part of the file... I was just trying to demonstrate the repetitive pattern. – SSDdude Sep 10 '18 at 13:23
2

How about this. Single run of awk, so likely quite fast in comparison to the original script.

$ awk -F/ 'BEGIN{print "DOM,    PROJ, CFG,      LOOKUP NAME VALUE(s)"}/source\/EDDG/{a=$2", "$3", "substr($8,0,length($8)-2)", "}/lookup_name/{gsub(/^.*value="/,"");gsub(/".*/,"");print a$0}' 4.txt
DOM,    PROJ, CFG,      LOOKUP NAME VALUE(s)
source, EDDG, test.cfg, CUSTOMER_1
source, EDDG, test.cfg, CODE_1
$

Or, formatted more nicely:

$ awk -F/ 'BEGIN {
            print "DOM,    PROJ, CFG,      LOOKUP NAME VALUE(s)"
           }
           /source\/EDDG/ {
            a=$2", "$3", "substr($8,0,length($8)-2)", "}
           /lookup_name/ {
            gsub(/^.*value="/,"")
            gsub(/".*/,"")
            print a$0
           }' 4.txt
DOM,    PROJ, CFG,      LOOKUP NAME VALUE(s)
source, EDDG, test.cfg, CUSTOMER_1
source, EDDG, test.cfg, CODE_1
$
  • 1
    +1 I see a lot of questions like this where people are trying to do something really convoluted (especially using sed) which turns out to be simple and straightforward using AWK. Many people just don't think of it or don't know AWK. Back in the day, I was a big SNOBOL4 fan, so when I discovered AWK, it was love at first sight. – Joe Sep 14 '18 at 22:03
  • 1
    Thx Steve and Joe... you are both right... awk RULES! This works flawlessly! – SSDdude Nov 27 '18 at 15:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.