I am trying to create a script which monitors the files sizes and grep the Job ID. The problem is the script returns only first ID even if there are multiple processes running. Any idea why this is not working?

find /opt/Logs -type f -size +10k > $filename
count='cat bigfiles | wc -l'

while read -r line

pid= more "$line" | grep ID | awk -F '=' '{print $2}'>> pid.txt

done < $filename

Also I need to scan multiple directories and the script should scan for the files if the size of the log file reaches 25% of file system utilization. Any suggestions on how I can incorporate this?

This works if I use for instead of while

 for line in $(cat bigfiles)                                                                                       
  do echo "$line"                                                                                                
  pid= grep JOBID "$line" | awk -F '=' '{print $2}' >> pid.txt                                  
  kill $(more $line | grep JOBID | awk -F '=' '{print $2}')                                    
  bhist -l $(more $line | grep JOBID | awk -F '=' '{print $2}') >> hist.txt;            

2 Answers 2


It's difficult to say why your script behaves badly since we don't know what the files that you are dealing with looks like. All we know is that you are interested in lines that contain the string ID and that you'd like to get whatever is after a = on those lines.

Your script contains two unused variables: count (a string containing a shell command) and pid (an empty variable). You also jump through quite a number of hoops to get to what's after =:

more "$line" | grep ID | awk -F '=' '{ print $2 }'

The more command is a pager, meaning it's a utility for viewing files. Instead of using that, you could let grep read the file directly:

grep ID "$line" | awk -F '=' '{ print $2 }'

But this is just letting grep do something that awk can do by itself,

awk -F '=' '/ID/ { print $2 }' "$line"

But we can do better than that. Instead of saving all the pathnames to the $filename file, we can let find do all the work for us:

find /opt/Logs -type f -size +10k \
    -exec awk -F '=' '/ID/ { print $2 }' {} + >pid.txt

This will find all regular files in or below /opt/Logs with size larger than 10 KB. For each such file, what's after (the first) = on each line containing the string ID will be printed. The result of that will be saved to pid.txt.


filename='bigfiles' find /opt/Logs -type f -size +10k > $filename

for line in $(cat bigfiles)

echo "$line"

pid= grep JOBID "$line" | awk -F '=' '{print $2}' >> pid.txt

kill $(more $line | grep JOBID | awk -F '=' '{print $2}')

bhist -l $(grep JOBID "$line" | awk -F '=' '{print $2}') >> hist.txt;

  • filename would be unset in the shell on the first line and you ought to get an error there when trying to redirect to nothing. Looping over the output of cat is ugly. You still have an empty pid variable defined for grep. I can't really see what this answer is trying to say.
    – Kusalananda
    Aug 1, 2018 at 8:28

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .