Shell Script not reporting correctly

Question

I am trying to create a script which monitors the files sizes and grep the Job ID. The problem is the script returns only first ID even if there are multiple processes running. Any idea why this is not working?

#!/bin/sh
filename='bigfiles'
find /opt/Logs -type f -size +10k > $filename
count='cat bigfiles | wc -l'

while read -r line
do

pid= more "$line" | grep ID | awk -F '=' '{print $2}'>> pid.txt

done < $filename

Also I need to scan multiple directories and the script should scan for the files if the size of the log file reaches 25% of file system utilization. Any suggestions on how I can incorporate this?

This works if I use for instead of while

 for line in $(cat bigfiles)                                                                                       
  do echo "$line"                                                                                                
  pid= grep JOBID "$line" | awk -F '=' '{print $2}' >> pid.txt                                  
  kill $(more $line | grep JOBID | awk -F '=' '{print $2}')                                    
  bhist -l $(more $line | grep JOBID | awk -F '=' '{print $2}') >> hist.txt;            
  done                                                                                                                

Answer 1

It's difficult to say why your script behaves badly since we don't know what the files that you are dealing with looks like. All we know is that you are interested in lines that contain the string ID and that you'd like to get whatever is after a = on those lines.

Your script contains two unused variables: count (a string containing a shell command) and pid (an empty variable). You also jump through quite a number of hoops to get to what's after =:

more "$line" | grep ID | awk -F '=' '{ print $2 }'

The more command is a pager, meaning it's a utility for viewing files. Instead of using that, you could let grep read the file directly:

grep ID "$line" | awk -F '=' '{ print $2 }'

But this is just letting grep do something that awk can do by itself,

awk -F '=' '/ID/ { print $2 }' "$line"

But we can do better than that. Instead of saving all the pathnames to the $filename file, we can let find do all the work for us:

find /opt/Logs -type f -size +10k \
    -exec awk -F '=' '/ID/ { print $2 }' {} + >pid.txt

This will find all regular files in or below /opt/Logs with size larger than 10 KB. For each such file, what's after (the first) = on each line containing the string ID will be printed. The result of that will be saved to pid.txt.

Related:

• Understanding the -exec option of `find`
• Why is looping over find's output bad practice?

Answer 2

filename='bigfiles' find /opt/Logs -type f -size +10k > $filename

for line in $(cat bigfiles)

do 
echo "$line"

pid= grep JOBID "$line" | awk -F '=' '{print $2}' >> pid.txt

kill $(more $line | grep JOBID | awk -F '=' '{print $2}')

bhist -l $(grep JOBID "$line" | awk -F '=' '{print $2}') >> hist.txt;

done