I have a few Python scripts laying around, and I'm working on rewriting them. I have the same problem with all of them.
It's not obvious to me how to write the programs so that they behave like proper unix tools.
$ cat characters | progname
$ progname characters
should produce the same output.
The closest thing I could find to that in Python was the fileinput library. Unfortunately, I don't really see how to rewrite my Python scripts, all of which look like this:
#!/usr/bin/env python # coding=UTF-8 import sys, re for file in sys.argv[1:]: f = open(file) fs = f.read() regexnl = re.compile('[^\s\w.,?!:;-]') rstuff = regexnl.sub('', fs) f.close() print rstuff
The fileinput library processes stdin if there is a stdin, and processes a file if there is a file. But it iterates over single lines.
import fileinput for line in fileinput.input(): process(line)
I really don't get that. I guess if you're dealing with small files, or if you're not doing much to the files, this may seem obvious. But, for my purposes, this makes it much slower than simply opening the entire file and reading it into a string, as above.
Currently I run the script above like
$ pythonscript textfilename1 > textfilename2
But I want to be able to run it (and its brethren) in pipes, like
$ grep pattern textfile1 | pythonscript | pythonscript | pythonscript > textfile2