1

I am currently working on script with the following goals:

  1. Script will check every 15-minute if there is AppA.log4 in /logs directory. There are numerous apps in /logs directory, for example AppA.log4, AppB.log4, AppC.log4
  2. If there is AppA.log4, it will be renamed to AppA.log.<timestamp>
  3. zip the renamed file and move to /backupDIR

So far, I have created working script however it is hardcoded.

Can you help improving this to be more dynamic? For example, AppA.log4 and AppB.log4 will be moved and zipped, leaving AppC.log4 in the /logs directory.

What if there's a separate file that lists all the App (AppA and AppB) that will be read by this script. How can I apply it here. Thank you in advance.

TIMESTAMP=`date "+%Y.%m.%d-%H.%M"`
LOGS="AppA.log.4"
APPLOGS_DIR="/logs/"
BACKUP_DIR="/backupDIR/"

cd $APPLOGS_DIR;

if [ -f "$LOGS" ];
then
        mv $LOGS $BACKUP_DIR;
                cd $BACKUP_DIR;
                mv $LOGS AppA_$TIMESTAMP;
        gzip AppA_$TIMESTAMP;
else
    echo "No log file to be backed up"
fi
10
  • use ls | find *.log4 | sed s/log4/log to dynamically use the names (also it will rename log4 to log) Jul 17 '20 at 7:00
  • Thank you for your response. Greatly appreciated. What I mean is, my goal now is to move and gzip AppA.log4 and AppB.log4. However there is still AppC.log4 in the /logs/ directory. I just want to specify AppA and AppB. I am thinking of another file where I can input AppA and AppB. The script will read this, but not yet sure on how can I implement it. Jul 17 '20 at 7:41
  • I think I understand what you want to do, but please edit your post to show the content of your /logs directory before and the desired content after running your script, so that we can understand which files should be treated in what way.
    – AdminBee
    Jul 17 '20 at 8:16
  • 2
    @theSwapnilSaste Please note that parsing the output of ls is highly disrecommended as it will stumble on spaces or other special characters in the filenames (even though it would appear that in this case, the names are "well-behaved" in that respect). Also, why do you pipe the output of ls into find?
    – AdminBee
    Jul 17 '20 at 8:18
  • 1
    Have you considered that the source log file(s) might be in use? (Something is still writing to them.) In which case once you have compressed them you could end up losing log data that's being written to the now-deleted log file.
    – roaima
    Jul 17 '20 at 8:57
2

you can use for loop and use string manipulation to generate output file name ${var%.*} based on input file name

TIMESTAMP=$(date "+%Y.%m.%d-%H.%M")
APPLOGS_DIR="./logs"
BACKUP_DIR="./backupDIR"

# canonicalize relative paths
APPLOGS_DIR=$(realpath "$APPLOGS_DIR")
BACKUP_DIR=$(realpath "$BACKUP_DIR")

# delete this line - just for demo
echo "AppA.log.4\nAppB.log.4" >> ./index.txt

# read file names from list
LOGS=$(sort ./index.txt | uniq) || exit 127

cd "$APPLOGS_DIR" || exit 127

for log in $LOGS
  do
    if [ -f "$log" ]
      then
        mv -f $log "$BACKUP_DIR"
        cd "$BACKUP_DIR" || exit 127
        mv $log ${log%.*}_${TIMESTAMP}
        gzip ${log%.*}_${TIMESTAMP}
        cd "$APPLOGS_DIR"
      else
        echo "No log file to be backed up"
    fi
done

However, just archive & remove file can be done without cd

while IFS='' read -r log || [ "$log" ]
  do
    [ -f "$APPLOGS_DIR/$log" ] && \
    gzip -nc "$APPLOGS_DIR/$log" > "$BACKUP_DIR/${log%.*}_${TIMESTAMP}.${log##*.}.gz" && \
    rm "$APPLOGS_DIR/$log" || \
    echo "No '$log' file to be backed up"
done <<< $(sort ./index.txt | uniq)

or as one-liner (when output file names doesn't matter and file names does not contain whitespaces) use the -S flag for suffix and process the index.txt straight from gzip without any loop

cd "$APPLOGS_DIR" && \
gzip -S .${TIMESTAMP}.gz $(cat $OLDPWD/index.txt) && \
mv -f *.${TIMESTAMP}.gz "$BACKUP_DIR"
0
0

I modified what you provided to add a for loop where you can specify the various log prefixes. This assumes you are calling this script every 15 minutes from a job scheduler such as cron or autosys, if not then it can be wrapped in a while true loop with a 15 minute sleep.

TIMESTAMP=`date "+%Y.%m.%d-%H.%M"`
LOGS="AppA AppB"
# to pull log prefixes from a file replace with the next line
#LOGS="$(cat prefixes.txt)"
LOGSUFFIX=".log.4"
APPLOGS_DIR="/logs"
BACKUP_DIR="/backupDIR"

for LOG in ${LOGS}
do
    if [ -f "${APPLOGS_DIR}/${LOG}${LOGSUFFIX}" ]
    then
        mv "${APPLOGS_DIR}/${LOG}${LOGSUFFIX}" "${BACKUP_DIR}/${LOG}_${TIMESTAMP}"
        gzip "${BACKUP_DIR}/${LOG}_${TIMESTAMP}"
    else
        echo "No ${LOG}${LOGSUFFIX} file to be backed up"
    fi
done
2
  • 2
    Is this an answer? It has Syntax errors, unquoted file paths and you give no explanation whatsoever. And also it does not help with OPs request to "help improving this to be more dynamic".
    – pLumo
    Jul 17 '20 at 8:41
  • 2
    upvoted because syntax error was just a missing quote and unquoted paths are no problem as long as not containing whitespaces
    – alecxs
    Jul 17 '20 at 14:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.