0

SunOS 5.8

Directory structure

/TEST/CHM

CHM
   A
     file1.txt
     file2.txt
   B
     file3.txt
   C
     file4.txt
     file5.txt
     file6.txt

If the parent directory CHM has less than 8 files/subdirectories zip it up normally. If the parent directory CHM has 8 or more files/subdirectories create a zip archive for ever 5 files. This is for testing only. In production it will be 10000 files, not 5. Parent directory CHM could have 0 to n subdirectories.

#!/bin/bash
set -e

cd $subdir/

# vars
num=8  # set 10000 in production

for dir in $subdir
do
    dir=${dir%*/}

    # testing code only
          if [[ ${dir##*/} = "CHM" ]]
          then
                numfile=$(ls * | wc -l)
          fi

          if [ "$numfile" -lt "$num" ]
          then
               zip -r -6 ${dir##*/}.zip .
          else
               ls * > files
               split -l 5 - files < files
               for i in files[a-z][a-z]; do
                   zip -6 "$i.zip" -@ < "$i"
               done
          fi
    # end test
done
exit


   zip warning: name not matched: A:
   zip warning: name not matched: file1.txt
   zip warning: name not matched: file2.txt
   zip warning: name not matched: B:

zip error: Nothing to do! (filesaa.zip)

The else part of the second if statement is failing and I'm not sure why. I'm looking to create:

CHM.zip master
CHM.001.zip
CHM.002.zip
CHM.003.zip

So I can unzip in the same order on a remote server.

0

I'm not entirely sure I followed your rules completely, but it might be easier to use standard utilities (and simplify the rules for what and when to create):

find . -type f | split -d -a3 -l $num --filter='zip -@ $FILE.zip' - "$pfx".

find is rather obvious, tweak options to your liking. You might want to sort the list before passing its output to split.

split:

  • - d - use numbering instead of alphabetical suffixes
  • -a3 - suffix length (the numbering) 3 places (digits in this case)
  • -l - number of lines after which to increase counter
  • --filter=COMMAND - pass lines to a command instead of writing into a file
  • COMMAND - $FILE is interpreted by split as a filename it would normally use for writing, which means it has to be single-quoted (or backslashed to prevent interpretation by the shell executing the above command line)
  • - - explicitly require parsing standard input
  • "$pfx". your preferred prefix. The dot is intentional, split doesn't append it automatically.
2
  • I get an error: find: path-list predicate-list split: illegal option -- d
    – Stringer
    Jun 22 '16 at 20:47
  • You may need a dot "." after the "find" command. Jun 22 '16 at 20:51
0

I have the solution that works on Sun Solaris. This will zip up 10000 files per each multipart zip and scp the master zip to the remote server.

#!/usr/bin/env bash
#
#------------------------------------------------------------
#-- Zip up content files based on $1 parameter
#------------------------------------------------------------
function zip_files {
   SRC="/ads/data02/CTS/Data/$1"
   #-- destination folder
   DST="/ads/acct/oracle/CTS"
   #-- amount of files that should go in each zip file
   FILES_PER_ZIP=10000

#-----------------------------------------------------------

  FILES="/tmp/multizip.$$"
  ZIP_PREFIX=$(basename $SRC)

#-- generate the list of the files to zip
  find $SRC > $FILES

#-- zip the files
  NUM=0
  ZIP_NUM=1
  printf -v ZIPFILE "$DST/$ZIP_PREFIX%03d" $ZIP_NUM
  while read -r line; do
    echo $line | sed -e 's/.*/"&"/' | xargs zip -6 $ZIPFILE
    ((NUM++))
    if [ $NUM -eq $FILES_PER_ZIP ]; then
       NUM=0
       ((ZIP_NUM++))
       printf -v ZIPFILE "$ZIP_PREFIX%03d" $ZIP_NUM
    fi
done < $FILES

#-- generate master zip file
/usr/bin/ls $DST/${ZIP_PREFIX}*.zip | xargs zip $DST/$ZIP_PREFIX

#-- perform cleanup

   function finish {
     /usr/bin/rm $FILES
     /usr/bin/rm /ads/acct/oracle/CTS/${ZIP_PREFIX}[0-9][0-9][0-9].zip
   }

   trap finish EXIT
}


#------------------------------------------------------------
#-- Main processing
#------------------------------------------------------------

for dir in /ads/data02/CTS/Data/*/
do
    d=$(basename $dir)
    zip_files $d
    scp "/ads/acct/oracle/CTS/${d}.zip" user1@10.10.20.25:/var/www/html/CTS/Content/A/TMP
    /usr/bin/rm -f "/ads/acct/oracle/CTS/${d}.zip"
done
exit

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.