we have a database that we take backup from it every night. backup files are 6 separate files in 6 separate directories. after backup is taken successfully, all parts will lftp to a remote server. the lftp commands are generated by a bash script which redirect output to a file named ftpfiles.sh then execute it. here is the content of ftpfiles.sh:
lftp -u user,pass 1.1.1.1 <<end_script
mkdir BackUp
cd BackUp
lcd /data10/customerBackup
put CUSTOMER.0.DBPART000.20220705010003.001
lcd /data1/customerBackup
put CUSTOMER.0.DBPART000.20220705010003.002
lcd /data2/customerBackup
put CUSTOMER.0.DBPART000.20220705010003.003
lcd /data7/customerBackup
put CUSTOMER.0.DBPART000.20220705010003.006
lcd /data8/customerBackup
put CUSTOMER.0.DBPART000.20220705010003.005
lcd /data9/customerBackup
put CUSTOMER.0.DBPART000.20220705010003.004
quit
end_script
exit 0
since each file is large, this process takes nearly 7 hours to be completed. so i wanted to send these files parallelly. if i want to use mirror
option in lftp which can send files parallelly, i should add many includes and excludes which is risky. other option is to send each put
process to background. here are the steps in command line (not using bash script):
[root@autodb /]# lftp -u user,pass 1.1.1.1
lftp [email protected].:~> mkdir BackUp
lftp [email protected].:~> cd BackUp
lftp [email protected].:~> lcd /data10/customerBackup
lftp [email protected].:~> put CUSTOMER.0.DBPART000.20220705010003.001
'CUSTOMER.0.DBPART000.20220705010003.001' at 40555088 (1%) [Sending data] {I PRESSED CTRL+Z IN HERE}
[0] put CUSTOMER.0.DBPART000.20220705010003.001 &
CUSTOMER.0.DBPART000.20220705010003.001 (8%) 45.85M/s eta:74s [Sending data]
lftp [email protected].:~> quit
[111] Moving to background to complete transfers...
[root@autodb /]#
i tried to achieve this in bash script but no luck. for example putting these in bash script:
(put CUSTOMER.0.DBPART000.20220705010003.001) &
put CUSTOMER.0.DBPART000.20220705010003.001 &
so is it possible to use ctrl+z in bash script to send lftp process to background?
also here is the script that generates lftp commands:
FTPFILES1='/data1/cronJobs/ftpfiles.sh'
DBNAME=CUSTOMER
##ftp files
cd /data1/cronJobs/
cat /dev/null > ftpfiles.sh
echo "lftp -u $FTPUSER,$FTPPASSWD $FTPSRV <<end_script
mkdir BackUp
cd Backup " >> $FTPFILES1
BACKUP_FILE_ARRAY=( /data*/"${DBNAME,,}"Backup/"$DBNAME.0.DBPART000".$(date +%Y%m%d)* )
for BACKUP_FILE in "${BACKUP_FILE_ARRAY[@]}"
do
echo "lcd $(dirname $BACKUP_FILE)" >> $FTPFILES1
echo "put $(basename $BACKUP_FILE)" >> $FTPFILES1
done
echo "quit
end_script
exit 0 " >> $FTPFILES1