0

I have to do comparison on weekends of last 5 days files from two different paths on a production server:

ABC/PROD
QRS/PROD 

The files have the same names but the timestamp format that follows is different, e.g.:

ABC/PROD/Archive

file1_20210902093041.bak   
other_file_20210902104556.bak  

QRS/PROD/Archive

file1_02092021094321.bak
other_file_02092021104545.bak

I have to copy all these files to a third location on the same server and do a sort and compare for the files which are generated and on same day and have same name.

Also there are n number of feeds generated everyday in both locations but I have to copy only certain 11 files and do comparison for each day of the week.

I want to understand the best approach and develop a shell script to automate this.

I am new to Shell scripting.

1
  • Your example may not include enough detail to permit a full response, particularly as regards the "n number of feeds" and "certain 11 files." Finally, although you say "The files have the same names but the timestamp format that follows is different," in your example, even after allowing for a change in the date format, the filenames themselves are different. Your question would likely be improved if you can spell out more clearly how you intend to match the filenames, and showing us manually the commands you would expect the script to perform to compare the files.
    – Jim L.
    Commented Sep 2, 2021 at 21:41

1 Answer 1

0

You have a problem with detecting correct date format. I will give you a base to start with

#!/bin/bash

sort_files() {
  local file dirname filename date_old date_new
  while read -r file
    do
      dirname="${file%/*}"
      filename="${file##*/}"
      date_old="${filename##*_}"
      date_old="${date_old%.*}"
      filename="${filename%_*}"
      case "$date_old" in
        # YYYYMMDDhhmmss
        202[0-9][0-1][0-9][0-3][0-9][0-2][0-9][0-5][0-9][0-5][0-9])
          date_new="$date_old"
        ;;
        # YYYYDDMMhhmmss
        202[0-9][0-3][0-9][0-1][0-9][0-2][0-9][0-5][0-9][0-5][0-9])
          date_new="${date_old:0:4}${date_old:6:2}${date_old:4:2}${date_old:8:6}"
        ;;
        # MMDDYYYYhhmmss
        [0-1][0-9][0-3][0-9]202[0-9][0-2[0-9][0-5][0-9][0-5][0-9])
          date_new="${date_old:4:4}${date_old:0:2}${date_old:2:2}${date_old:8:6}"
        ;;
        # DDMMYYYYhhmmss
        [0-3][0-9][0-1][0-9]202[0-9][0-2][0-9][0-5][0-9][0-5][0-9])
          date_new="${date_old:4:4}${date_old:2:2}${date_old:0:2}${date_old:8:6}"
        ;;
        *)
          continue
        ;;
      esac
      printf '%s\t%s\t%s:\t%s\n' "$filename" "${date_new:0:8}" "${dirname%%/*}" "$file"
  done < <(find $dirs -iregex ".*/$searchdir/[^/]*_[0-9]+\.$suffix") | sort
  return 0
}

# set global vars for find regex
dirs='ABC QRS'
searchdir=Archive
suffix=bak

# call function, output columns 2 4
sort_files | cut -f2,4

# call function, diff two files
sort_files | cut -f1,2,4 | while IFS=$'\t' read -r filename date file
  do
    if [ "$filename" = "$filename2" ]
      then
        if [ "$date" = "$date2" ]
          then
            diff -sq "$file2" "$file" || diff -y "$file2" "$file"
            echo -e '\n'
        fi
    fi
    filename2="$filename"
    date2="$date"
    file2="$file"
done

exit 0

do whatever you want to do, use the function and cut desired columns as input for your array, loop, diff, rename whatever

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .