I have a python script

$ cat ~/script.py
import sys
from lxml import etree
from lxml.html import parse
doc = parse(sys.argv[1])
title = doc.find('//title')
title.text = span2.text.strip()
print etree.tostring(doc)

I can run the script on an individual file by issuing something like

$ python script.py foo.html > new-foo.html

My problem is that I have a directory ~/webpage that contains hundreds of .html files scattered throughout sub-directories. I would like to run ~/script.py on all of these html files. How can I do this?

I'm aware that I can list all the .html files under ~/webpage/ by issuing

$ find ~/webpage/ -name "*.html"

but I'm not quite sure how to use this list to run my script on them.

  • Python can walk a directory no problem, check out os.walk()
    – iruvar
    Commented Jan 12, 2016 at 2:48

2 Answers 2


Use the -exec command of find

find ~/webpage/ -name "*.html" -exec sh -c 'python script.py {} > new-{}' \;

For some versions, you might need to do

find ~/webpage/ -name "*.html" -exec sh -c 'python script.py $0 > new-$0' {} \;
  • Double check before you run this that it actually works as expected. I believe some versions of find require {} to be a single arg, or they won't expand it.
    – Wildcard
    Commented Jan 12, 2016 at 2:33

You can try a for loop to iterate through the files in the directory:

for f in *.html; do python script.py f > new-"$f".html; done

  • This will cause every iteration of the loop to overwrite new-foo.html. Should be ... > new-"$f".html.
    – gardenhead
    Commented Jan 12, 2016 at 3:24
  • I updated my answer
    – ryekayo
    Commented Jan 12, 2016 at 3:25

Not the answer you're looking for? Browse other questions tagged .