I have written a shell script which has to do the following:
- Capture session commands into one file.
- Every individual command into separate file.
- Mail every individual command file content based on certain criteria.
As for my observation, the loop has to iterate minimum of 25,000 times. Now my problem is that it is taking more than 6 hours to complete all iterations.
Below is the main part of the script which is taking long time to process.
if [ -s "$LOC/check.txt" ]; then
while read line; do
echo -e " started processing $line at `date` " >> "$SCRIPT_LOC/running_status.txt"
TST=`grep -w $line $PERM_LOC/id_processing.txt`
USER=`echo $TST | grep -w $line | awk -F '"' '{print $10}'`
HOST=`echo $TST | grep -w $line | awk -F '"' '{print $18}'`
ID=`echo $TST | echo $line | tr -d '\"'`
IP=`echo $TST | grep -w $line | awk -F '"' '{print $20}'`
DB=`echo $TST | grep -w $line | awk -F '"' '{print $22}'`
CONN_TSMP=`echo $TST | grep -w $line | awk -F '"' '{print $2}'`
if [ -z "$IP" ]; then
IP=`echo "$HOST"`
fi
if [ "$USER" == "root" ] && [ -z $DB ]; then
TARGET=/data1/sessions/root_sec
CMD_TARGET=/data1/commands/root_commands
FILE=`echo "$ID-$CONN_TSMP-$USER@$IP.txt"`
else
TARGET=/data1/sessions/user_sec
CMD_TARGET=/data1/commands/user_commands
FILE=`echo "$ID-$CONN_TSMP-$USER@$IP.txt"`
fi
ls $TARGET/$FILE
If [ $? -ne 0 ]; then
echo $TST | awk -F 'STATUS="0"' '{print $2}'| sed "s/[</>]//g" >> "$TARGET/$FILE"
echo -e "\n" >> "$TARGET/$FILE"
fi
grep $line $LOC/out.txt > "$LOC/temp.txt"
while read val; do
TSMP=`echo "$val" | awk -F '"' '{print $2}'`
QUERY=`echo "$val" | awk -F 'SQLTEXT=' '{print $2}' | sed "s/[/]//g"`
echo " TIMESTAMP=$TSMP " >> "$TARGET/$FILE"
echo " QUERY=$QUERY " >> "$TARGET/$FILE"
RES=`echo "$QUERY" | awk {'print $1'} | sed 's/["]//g' `
TEXT=`grep "$RES" "$PERM_LOC/commands.txt"`
if [ -n "$TEXT" ]; then
NUM=`expr $NUM + 1`
SUB_FILE=`echo "$ID-$command-$NUM-$TSMP-$USER@$IP.txt"`
echo -e "===============\n" > "$CMD_TARGET/$SUB_FILE"
echo "FILE = \"$SUB_FILE\"" >> "$CMD_TARGET/$SUB_FILE"
### same way append 6 more lines to $SUB_FILE
SUB=`echo "$WARN_ME" | grep "$command"`
if [ "$command" == "$VC" ]; then
STATE=`echo " very critical "`
elif [ -z "$SUB" ]; then
STATE=CRITICAL
else
STATE=WARNING
fi
if [ "$USER" != "root" -a "$command" != "$VC" ]; then
mail command &
elif [ "$USER" == "root" -a -z "$HOST" ]; then
mail command &
elif [ "$USER" == "root" -a "$command" == "$VC" ]; then
mail command &
else
echo -e "some message \n" >> $LOC/operations.txt
fi
fi
done < "$LOC/temp.txt"
done < "$LOC/check.txt"
fi
Can any one help me how to optimize this code either by dividing or by changing logic or by using functions or by anything else?
Here I have to use a shell script only and the server on which the script will be executed should not take more than 3GB of RAM to process it.
Any help is very very useful.