unable to extract file which is uploaded to FTP server

Question

Source System:

• CentOS 7
• bash shell
• zip

Destination System:

• FTP server
• CentOS 6.4
• bash shell
• unzip

I wrote a script to archive file and send it to FTP server using shell script

#!/bin/bash

# Declare no. of days
days=15

# Declare Source path of sql files and Destination path of backup directory
dumps=/home/applications/backup

bkpdir=/home/applications/backup/olddumps

# Find sql dumps of ets
files=($(find $dumps/*.sql -mtime +"$days"))

for file in ${files[*]}

do
# Move each file into backup dir which is 15 days old
echo "file is: $file\n";

mv $file $bkpdir

# Find the sql files and compress them

cd $bkpdir

filename=$(basename $file)

zip $bkpdir/$filename.zip $filename

# FTP Login

HOST=a.b.c.d

USER=xxxx

PASS=yyyyy

REM_DIR=/olddumps/sqlfiles

echo "Uploading file via FTP:"

ftp -in $HOST <<EOF

quote USER $USER

quote PASS $PASS

cd $REM_DIR

put $filename.zip

bye

EOF

# Remove sql files if any
rm $bkpdir/$filename

done

# Remove compressed files which are 6 months old
find $bkpdir/*.zip -type f -mtime +180 -exec rm {} \;

Now the problem is the compressed file in destination system is not getting extracted using unzip command and shows the following error:

Archive: emt_bus-08-09-16-03-29.sql.zip

caution: zipfile comment truncated
error [emt_bus-08-09-16-03-29.sql.zip]: missing 49666528 bytes in zipfile
(attempting to process anyway)
error [emt_bus-08-09-16-03-29.sql.zip]: start of central directory not found;
zipfile corrupt.
(please check that you have transferred or created the zipfile in the
appropriate BINARY mode and that you have compiled UnZip properly)

I used tar to archive, but no luck. It does not extract file in destination system and shows following error

gzip: stdin: invalid compressed data--format violated
emt_bus-08-09-16-03-29.sql
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now

How to fix the issue?

Answer 1

Likely you already have error messages you're just not seeing when you run that script.

Capture all output to a log by executing it with
script.sh > log.txt 2>&1

The last part redirects stderr to stdout, in this case log.txt

Probably what you'll find is that you're timing out (how many files are you sending, how stable is your connection), or maybe that a command you think waits for the next command doesn't actually wait in the script.

Answer 2

Your script should set binary (after the quote commands). Otherwise, it will use text-mode for this script.

If it were interactive, the ftp client which I just tested will assume binary mode. But not when it is non-interactive.

This is what I tested/and fixed:

#!/bin/sh
ftp -in invisible-island.net <<EOF
quote USER anonymous
quote PASS anonymous
binary
cd cproto
get cproto.tar.gz
bye
EOF