Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I want to write a Makefile to find specific files and run a Python script on each file. The Python script accepts sys.stdin input.

find $(W)/$(OVE) -name "*.xml" -print | \
while read x ; do \
  cat $x | /opt/exp/bin/python2.7 process_results.py > $(W)/$(OVE)/$(dirname $x)_$(basename $x).xml \
done

The output is $(dirname $x)_$(basename $x).xml file which is an empty _.xml file.

When I run this command on command line, it works properly but in Makefile it doesn't work.

Can you help me what is wrong with this command?

share|improve this question
    
Also, what is the exact output from your make invocation? (You may want to do this in a directory with only one or a few files, to minimize clutter.) –  Michael Kjörling Nov 24 '12 at 11:57
    
I want to run a python script on a few xml files in the directory and save the results in a new xml file –  Hamideh Nov 24 '12 at 13:55

3 Answers 3

up vote 2 down vote accepted

First, you need to quote the dollar signs that are supposed to be interpreted by the shell. As it is, $x and $(dirname $x) and $(basename $x) are parsed by make. You're also missing a semicolon at the end of the body of the while loop. Check the output from running make, you should see the shell complaining about that.

find $(W)/$(OVE) -name "*.xml" -print | \
while read x ; do \
  cat $$x | /opt/exp/bin/python2.7 process_results.py > $(W)/$(OVE)/$$(dirname $$x)_$$(basename $$x).xml; \
done

Another likely problem is that $(dirname $x) already contains the $(W)/$(OVE) part. For example, if $(W) is foo and $(OVE) is bar and $x is subdir/subsubdir/wibble.xml, you'll end up trying to write files like foo/bar/foo/bar/subdir/subsubdir_wibble.xml.xml, whereas you probably mean to write foo/bar/foo/bar/subdir/subsubdir_wibble.xml.xml. It's also weird that you're transforming a .xml file into a .xml.xml file. If you meant to strip the original .xml extension, you need to write basename $$x .xml, but it's pointless to remove the extension only to add it again. So you probably meant to write:

find $(W)/$(OVE) -name "*.xml" -print | \
while read x ; do \
  /opt/exp/bin/python2.7 process_results.py <$$x >$$(dirname $$x)_$$(basename $$x); \
done

A further problem is that if any of the runs of process_results.py fails, make will continue running and still report a successful build. Tell the shell to stop if an error occurs:

set -e; \
find $(W)/$(OVE) -name "*.xml" -print | \
while read x ; do \
  /opt/exp/bin/python2.7 process_results.py <$$x >$$(dirname $$x)_$$(basename $$x); \
done
share|improve this answer
    
when I use /opt/exp/bin/python2.7 process_results.py <$$x >$$(dirname $$x)_$$(basename $$x); the output is an empty file. It seems no input has been passed th the script, the script reads input from stdin. how can I change this line? thanks –  Hamideh Nov 25 '12 at 12:07
    
@Hamideh Works for me. Are you sure your script is producing output? Add a trace whenever it reads a line or something. –  Gilles Nov 25 '12 at 19:03

You need to quote the dollar-signs that you want to pass through to the shell, otherwise make will try to interpret them.

-find $(W)/$(OVE) -name -print | \
  while read x ; do \
      cat $$x | /opt/exp/bin/python2.7 process_results.py \
        > $(W)/$(OVE)/$$(dirname $$x)_$$(basename $$x).xml; \
  done

The single dollar-signs are make variable expansions. The double dollar-signs are passed through to the command line as a single dollar-sign.

share|improve this answer
    
Thanks for your reply Camh. But when I use $$x, in this case, even empty _.xml file is not created. –  Hamideh Nov 24 '12 at 14:03

$x, $(dirname $x) etc... will all get mangled by make as it interprets them as its own variables. Best to wrap this in a one-line shellscript and call that from the Makefile.

share|improve this answer
    
Thanks Dennis, I'm new to scripting and makefile. Can you give me a straight forward guideline. Thanks. –  Hamideh Nov 24 '12 at 14:05
    
@Hamideh. What Dennis is suggesting is to get your script to work as a pure bash .sh file first. Then take the additional step of embedding it in the Makefile. I recommend this approach too. 'get it working, then get it right, then get it fast'. –  Clayton Stanley Nov 27 '12 at 4:23

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.