0

HylaFax keeps a history of faxes sent for about a week in its spool folder. I have a script that can go through the files, extract the user and page counts and output it to a log.

What's the most efficient way to process the files in the folder on a daily basis without resulting in duplicate outputs in the log file and without moving/altering the processed files?

Description of Folder:

# ls /var/spool/hylafax/doneq/
q10000 q10001 ... q10010

Script

#!/bin/bash
for i in /var/spool/hylafax/doneq/q*
  do
   user=$(cat $i | grep "mailaddr" | sed 's/mailaddr://g')
   pgs=$(cat $i | grep "npages" | sed 's/npages://g')
      echo "$i $user - $pgs pages" >> /scripts/log_output.txt
done
2
  • What are duplicate outputs?
    – cuonglm
    May 6, 2015 at 15:50
  • If I set the above script to run daily, as is, it will process the entire folder daily, which means that most of the files will be reprocessed each day, resulting in duplicate output to the log file.
    – RyanH
    May 6, 2015 at 16:04

2 Answers 2

0

You asked

What's the most efficient way to process the files in the folder on a daily basis without resulting in duplicate outputs in the log file and without moving/altering the processed files?

One way to achieve this is to store a sorted list of the files that have been processed. Using comm against the list of the candidate files for processing will remove duplicates.

As an illustration of the technique, something like this could be used as a basis for handling for well-formed filenames such as those produced by HylaFax:

find * -print | sort > /tmp/current_files

test -f /tmp/previous_files || { echo "Come back tomorrow"; exit 0; }
comm -13 /tmp/previous_files /tmp/current_files > /tmp/new_files

# ... Process entries in /tmp/new_files ...

mv -f /tmp/current_files /tmp/previous_files
1
  • That looks like it should do the trick, so I'll take that as an answer.
    – RyanH
    May 6, 2015 at 17:11
0

This may be a less than elegant solution, but this is how I came up with a work around:

#!/bin/bash

processed=$(cat /scripts/processed_log.txt)

for i in /var/spool/hylafax/doneq/q*
  do
   case "${processed[@]}" in
       *"$i"*) ;;
       *) echo "New! Going to add $i to the log"
          echo $i >> /scripts/processed_log.txt
          user=$(cat $i | grep "mailaddr" | sed 's/mailaddr://g')
          pgs=$(cat $i | grep "npages" | sed 's/npages://g')
          echo "$i $user - $pgs pages" >> /scripts/log_output.txt
       ;;
esac
done

I just need to add logic to remove files in the process_log.txt that no longer exist in the spool dir. Sed should probably do the trick fine.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .