Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a few Python scripts laying around, and I'm working on rewriting them. I have the same problem with all of them.

It's not obvious to me how to write the programs so that they behave like proper unix tools.

Because this

$ cat characters | progname

and this

$ progname characters

should produce the same output.

The closest thing I could find to that in Python was the fileinput library. Unfortunately, I don't really see how to rewrite my Python scripts, all of which look like this:

#!/usr/bin/env python                                                                                                                                                                                                                                                         
# coding=UTF-8

import sys, re

for file in sys.argv[1:]:                                                                                                                                                                                                                                                     
    f = open(file)
    fs = f.read()
    regexnl = re.compile('[^\s\w.,?!:;-]')
    rstuff = regexnl.sub('', fs)
    f.close()
    print rstuff

The fileinput library processes stdin if there is a stdin, and processes a file if there is a file. But it iterates over single lines.

import fileinput
for line in fileinput.input():
    process(line)

I really don't get that. I guess if you're dealing with small files, or if you're not doing much to the files, this may seem obvious. But, for my purposes, this makes it much slower than simply opening the entire file and reading it into a string, as above.

Currently I run the script above like

$ pythonscript textfilename1 > textfilename2

But I want to be able to run it (and its brethren) in pipes, like

$ grep pattern textfile1 | pythonscript | pythonscript | pythonscript > textfile2
share|improve this question
4  
add comment

3 Answers

up vote 6 down vote accepted

Why not just

files = sys.argv[1:]
if not files:
    files = ["/dev/stdin"]

for file in files:
    f = open(file)
    ...
share|improve this answer
add comment

Check if a filename is given as an argument, or else read from sys.stdin.

Something like this:

if sys.argv[1]:
   f = open(sys.argv[1])
else:
   f = sys.stdin 

It's similar to Mikel's answer except it uses the sys module. I figure if they have it in there it must be for a reason...

share|improve this answer
    
What if two file names are specified on the command line? –  Mikel Sep 4 '12 at 21:28
3  
Oh absolutely! I didn't bother showing it because it was already shown in your answer. At some point you have to trust the user to decide what she needs. But feel free to edit if you believe this is best. My point is only to replace "open(/dev/stdin") with sys.stdin. –  rahmu Sep 4 '12 at 21:40
add comment

My preferred way of doing it turns out to be... (and this is taken from a nice little Linux blog called Harbinger's Hollow)

#!/usr/bin/env python

import argparse, sys

parser = argparse.ArgumentParser()
parser.add_argument('filename', nargs='?')
args = parser.parse_args()
if args.filename:
    string = open(args.filename).read()
elif not sys.stdin.isatty():
    string = sys.stdin.read()
else:
    parser.print_help()

The reason why I liked this best is that, as the blogger says, it just outputs a silly message if accidentally called without input. It also slots so nicely into all of my existing Python scripts that I have modified them all to include it.

share|improve this answer
2  
Sometimes you do want to enter the input interactively from a tty; checking isatty and bailing out does not conform to the philosophy of Unix filters. –  musiphil Sep 24 '13 at 7:58
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.