I have found another method which totally negates this post but I will try to illustrate what I did....
So I have a client config file delimited by '%'. This is read, line by line by a sftp process that will push and pull data to/from the clients. It looks something like this:
...
client_1%FileContent_1%xml%...(client connection info)
client_1%FileContent_2%txt%...(client connection info)
client_1%FileContent_3%pdf%...(client connection info)
client_2%FileContent_1%xml%...(client connection info)
client_2%FileContent_2%pdf%...(client connection info)
...
So my purge process reads this and looks at the first three fields. They are used to assemble base-path for each client.
/xmit/client/outbound
Now the above folders each have a 'sent' or 'received' and 'log' folders.
/xmit/client/outbound/sent
/xmit/client/outbound/log
And this sftp process sends and receives files for these customers and when each file is processed, it puts said file in the /sent & /received folders and a full trace on the send/receive of said file go into the /log folders.
So the purge process that I am writing reads the same config file to locate the folders to purge from but i want to consolidate the log files of the purge for each client.
#
# Inits....
#
basepath=/xmit
workingdir=$basepath/bin/Purge
purgeProcessLogFile=$workingdir/outboundPurge.log
#
cnfgFl=/xmit/bin/OutBoundScripts/clientsftpconfigoutbound.txt
# Config File Ref
#
declare -i cnfgFlNbrOfLns=`wc -l "$cnfgFl" | cut -d ' ' -f 1`
# Count lines in config file...
#
# Set beginning line and work from beginning line to ending line...
#
firstpass=1
declare -i cnfgFlStrtLn="53"
declare -i currentLnNbr=$cnfgFlStrtLn
lastClient='0'
#
# Begin while loop...
#
while [ "$currentLnNbr" -le "$cnfgFlNbrOfLns" ]
do
#
# Set what line to read from and read the data in...
#
option="$currentLnNbr"p
cnfgFlLn=`sed -n $option $cnfgFl`
#
# Increment counter for the next loop, you've allredy got the data...
#
declare -i currentLnNbr=`expr $currentLnNbr + 1`
#
# Check if very first character in config line is not a "#" and proceed.
#
fld=`echo $cnfgFlLn | cut -c1-1`
#
if [ "$fld" != \# ]
then
#
# Get file content... (ex: '7501' )
#
cntnt=`echo $cnfgFlLn | cut -d "%" -f 2`
#
# Extract account from
#
currClient=`echo $cnfgFlLn | cut -d "%" -f 1`
#
if [ $currClient != \# ]
then
#
# Check if on new account. If new account, set 'lastClient' to current account
# and reset log files...
#
if [ "$lastClient" != "$currClient" ]
then
#
lastClient=$currClient
#
# Set current date and time stamp...
#
dt=`date +%Y%m%d``date +%H%M%S`
#
# Set sent purge log...
#
sentPurgeFolder=$basepath/$currClient/outbound/sent
sentPurgeLogFile=$sentPurgeFolder/SentPurgeLogFile\_$dt.log
#
# Set sent log purge log...
#
sentLogPurgeFolder=$basepath/$currClient/outbound/logs
sentLogPurgeLogFile=$sentLogPurgeFolder/SentLogPurgeLogFile\_$dt.log
#
# Prime new logfiles...
#
cat /dev/null > $sentPurgeLogFile
cat /dev/null > $sentLogPurgeLogFile
fi
#
# See if there are any files to purge...
#
declare -i cntntFilesToPurge=`find $sentPurgeFolder -mtime +7 -print | wc -l`
if [ "$cntntFilesToPurge" -gt "0" ]
then
#
# 'cd' to sent folder, purge and upate logfile...
#
cd $sentPurgeFolder
dt1=`date +%Y%m%d``date +%H%M%S`
echo $dt1 - Purging "$cntntFilesToPurge" "'"$cntnt"'" data files that are 7 days or older from $sentPurgeFolder >> $purgeProcessLogFile
find $sentPurgeFolder -mtime +7 -print -delete >> $sentPurgeLogFile 2>&1
else
dt1=`date +%Y%m%d``date +%H%M%S`
echo $dt1" - No ""'"$cntnt"'" data files that are 7 days or older to purge from $sentPurgeFolder >> $purgeProcessLogFile
fi
#
# See if there are any logs to purge...
#
declare -i logFilesToPurge=`find $sentLogPurgeFolder -mtime +7 -print | wc -l`
if [ "$logFilesToPurge" -gt "0" ]
then
#
# 'cd' to logs folder, purge and upate logfile...
#
cd $sentLogPurgeFolder
dt1=`date +%Y%m%d``date +%H%M%S`
echo $dt1 - Purging "'"$cntnt"'" 'purge log' data are 7 days or older from $sentPurgeFolder >> $purgeProcessLogFile
find $sentLogPurgeFolder -mtime +1 -print -delete >> $sentLogPurgeLogFile 2>&1
else
dt1=`date +%Y%m%d``date +%H%M%S`
echo $dt1" - No logfiles for ""'"$cntnt"'" data that are 7 days or older to purge from $sentPurgeFolder >> $purgeProcessLogFile
fi
fi
fi
done
#
# E N D O F F I L E
#
The section :
declare -i cntntFilesToPurge=`find $sentPurgeFolder -mtime +7 -print | wc -l`
if [ "$cntntFilesToPurge" -gt "0" ]
then
cd $sentPurgeFolder
dt1=`date +%Y%m%d``date +%H%M%S`
echo $dt1 - Purging "$cntntFilesToPurge" "'"$cntnt"'" data files that are 7 days or older from $sentPurgeFolder >> $purgeProcessLogFile
find $sentPurgeFolder -mtime +7 -print -delete >> $sentPurgeLogFile 2>&1
else
dt1=`date +%Y%m%d``date +%H%M%S`
echo $dt1" - No ""'"$cntnt"'" data files that are 7 days or older to purge from $sentPurgeFolder >> $purgeProcessLogFile
fi
#
# 'cd' to logs folder, purge and upate logfile...
#
declare -i logFilesToPurge=`find $sentLogPurgeFolder -mtime +7 -print | wc -l`
if [ "$logFilesToPurge" -gt "0" ]
then
cd $sentLogPurgeFolder
dt1=`date +%Y%m%d``date +%H%M%S`
echo $dt1 - Purging "'"$cntnt"'" 'purge log' data are 7 days or older from $sentPurgeFolder >> $purgeProcessLogFile
find $sentLogPurgeFolder -mtime +1 -print -delete >> $sentLogPurgeLogFile 2>&1
else
dt1=`date +%Y%m%d``date +%H%M%S`
echo $dt1" - No logfiles for ""'"$cntnt"'" data that are 7 days or older to purge from $sentPurgeFolder >> $purgeProcessLogFile
fi
Was before:
#
# 'cd' to sent folder, purge and upate logfile...
#
cd $sentPurgeFolder
dt1=`date +%Y%m%d``date +%H%M%S`
find $sentPurgeFolder -mtime +7 -print -delete >> $sentPurgeLogFile 2>&1
#
# 'cd' to logs folder, purge and upate logfile...
#
cd $sentLogPurgeFolder
dt1=`date +%Y%m%d``date +%H%M%S`
find $sentLogPurgeFolder -mtime +1 -print -delete >> $sentLogPurgeLogFile 2>&1
This has satisfied my needs in terms of a 'purge' log.