1

I'm having some difficulty understanding why a particular process is failing. Basically, I have a line in my script that is listing directory contents, using tee to redirect the output to two files:

retval=`ls -1 $SOURCE_DIR/${2}/${3} | tee ${COPY_LIST} ${MOVE_LIST}`

The command works fine on the command line and both files are populated (around 8K of lines). The strange thing is that through cron, the script just "hangs"and checking the file only a partial list of the same directories is created (approx 3k records). Last line is also partial and not written completely. Not sure whether the problem is within tee in itself.

    #!/bin/ksh
#DAILY RUN

#DEBUG_SCREEN=ON
LOG_FILE=/daily_archive_v2_`date +%d-%m-20%y`.log
. /stdlib.ksh

##the datetimestamp for the file that contains the list of files to be archived
LIST_TIMESTAMP=`perl -e 'use POSIX qw(strftime); print strftime "%Y%m%d_%H%M",localtime(time());'`

Month_value=`date +%m`
typeset -i Prev_Month_value_tmp=`date +%m`
Prev_Month_value_tmp=${Prev_Month_value_tmp}-1
Year_value=`date +%y`

if [[ "${Month_value}" -eq 01 ]] then
  echo "Month is January"
  typeset -i Prev_Year_value=${Year_value}-1
Prev_Month_value_tmp=12
else
  Prev_Year_value="${Year_value}"
fi

if [[ "${Prev_Month_value_tmp}" -le 10 ]] then
  echo "Prev month is less than 10"
  Prev_Month_value="0${Prev_Month_value_tmp}"
else
  Prev_Month_value="${Prev_Month_value_tmp}"
fi
echo Previous Month Value = $Prev_Month_value

echo LOGFILE NAME SET TO $LOG_FILE
echo "DAILY REPORT [Archiving Month ${Month_value}]"
log "DAILY REPORT [Archiving Month ${Month_value}]"
log_line
SOURCE_DIR=/test
DEST_DIR=/Archives/month_"${Month_value}"_20"${Year_value}"
BACKUP_DIR=/vault/month_"${Month_value}"_20"${Year_value}"
mkdir -p $DEST_DIR
mkdir -p $BACKUP_DIR

COPY_LIST="${LIST_TIMESTAMP}_output_archiving.copy.lst"
MOVE_LIST="${LIST_TIMESTAMP}_output_archiving.move.lst"

#PARAMS : NAME, DIR, MASK
function move_files
{
  DEST_DIR=/month_"${Month_value}"_20"${Year_value}"
  FILES_COUNT=`ls -1 $SOURCE_DIR/$2/$3 | wc -l`
  if [ "${FILES_COUNT}" != "0" ]; then
    log "MOVE $FILES_COUNT $1 FILES $SOURCE_DIR/$2/$3 to $DEST_DIR/$2"
    log "MOVE $FILES_COUNT $1 FILES $SOURCE_DIR/$2/$3 to $BACKUP_DIR/$2"
    mkdir -p $DEST_DIR/$2
    mkdir -p $BACKUP_DIR/$2
retval=`ls -1 $SOURCE_DIR/${2}/${3} | tee ${COPY_LIST} ${MOVE_LIST}`
#echo retval=`ls -1 $SOURCE_DIR/${2}/${3} | tee ${COPY_LIST} ${MOVE_LIST}`
#       while read line
#         do
#         CURRENT_FILE=`echo $line`
#         #echo ${CURRENT_FILE}
#         cp -p ${CURRENT_FILE} $BACKUP_DIR/$2
#         mv ${CURRENT_FILE} $DEST_DIR/$2
#         log "File ${CURRENT_FILE} copied to $BACKUP_DIR/$4/"
#         log "File ${CURRENT_FILE} moved to $DEST_DIR/$4/"
#         done < ${COPY_LIST}
  else
    echo Nothing to move
  fi
#        gzip  $DEST_DIR/$2/*
#         gzip $BACKUP_DIR/$2/*
}

move_files  "TESTDIR" "dir2/test" "*.txt"

\rm -f $RESULTS_FILE
\rm -f $FILE_LIST
#rm -f $LOG_FILE
  • 2
    How is the script invoked from your crontab and what are the values of the various variables? – Kusalananda Feb 22 '17 at 8:46
  • Add an echo to the beginning of the line and redirect the script's output to a file (in the crontab, write /path/to/script > /tmp/script.log), then see what command you are actually running. – terdon Feb 22 '17 at 8:51
  • @Kusalananda - the values are just paths to the files. Variables are correctly set since both file lists are created but with incomplete data, so its not an issue with the variables. – Etienne Feb 22 '17 at 9:02
  • @terdon no errors are returned, added the echo and only the normal script loggin/echos are returned. I will try and use find maybe its an issue with the long list of files that are returned with ls maybe that is messing things out – Etienne Feb 22 '17 at 9:07
  • @Etienne that's one possibility, but I doubt it's the number of files returned, parsing ls is almost always a very bad idea. But please edit your question and show us the exact command being run by cron. We need to see what ls -1 $SOURCE_DIR/${2}/${3} | tee ${COPY_LIST} ${MOVE_LIST} is expanded to when run by cron so please add echo "ls -1 $SOURCE_DIR/${2}/${3} | tee ${COPY_LIST} ${MOVE_LIST}" to your script and add the output to your question. – terdon Feb 22 '17 at 9:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.