0

we have a database that we take backup from it every night. backup files are 6 separate files in 6 separate directories. after backup is taken successfully, all parts will lftp to a remote server. the lftp commands are generated by a bash script which redirect output to a file named ftpfiles.sh then execute it. here is the content of ftpfiles.sh:

   lftp -u user,pass 1.1.1.1  <<end_script
   mkdir BackUp
   cd BackUp
   lcd /data10/customerBackup
   put CUSTOMER.0.DBPART000.20220705010003.001
   lcd /data1/customerBackup
   put CUSTOMER.0.DBPART000.20220705010003.002
   lcd /data2/customerBackup
   put CUSTOMER.0.DBPART000.20220705010003.003
   lcd /data7/customerBackup
   put CUSTOMER.0.DBPART000.20220705010003.006
   lcd /data8/customerBackup
   put CUSTOMER.0.DBPART000.20220705010003.005
   lcd /data9/customerBackup
   put CUSTOMER.0.DBPART000.20220705010003.004
   quit
   end_script
   exit 0

since each file is large, this process takes nearly 7 hours to be completed. so i wanted to send these files parallelly. if i want to use mirror option in lftp which can send files parallelly, i should add many includes and excludes which is risky. other option is to send each put process to background. here are the steps in command line (not using bash script):

  [root@autodb /]# lftp -u user,pass 1.1.1.1
  lftp [email protected].:~> mkdir BackUp
  lftp [email protected].:~> cd BackUp
  lftp [email protected].:~> lcd /data10/customerBackup
  lftp [email protected].:~> put CUSTOMER.0.DBPART000.20220705010003.001
  'CUSTOMER.0.DBPART000.20220705010003.001' at 40555088 (1%) [Sending data] {I PRESSED CTRL+Z IN HERE}
   [0] put CUSTOMER.0.DBPART000.20220705010003.001 &
    CUSTOMER.0.DBPART000.20220705010003.001 (8%) 45.85M/s eta:74s [Sending data]
  lftp [email protected].:~> quit
  [111] Moving to background to complete transfers...
  [root@autodb /]#

i tried to achieve this in bash script but no luck. for example putting these in bash script:

  (put CUSTOMER.0.DBPART000.20220705010003.001) &
   put CUSTOMER.0.DBPART000.20220705010003.001 &

so is it possible to use ctrl+z in bash script to send lftp process to background?
also here is the script that generates lftp commands:

  FTPFILES1='/data1/cronJobs/ftpfiles.sh'
  DBNAME=CUSTOMER
  ##ftp files
  cd /data1/cronJobs/
  cat /dev/null > ftpfiles.sh
  echo "lftp -u $FTPUSER,$FTPPASSWD $FTPSRV  <<end_script
  mkdir BackUp
  cd Backup " >> $FTPFILES1
  BACKUP_FILE_ARRAY=( /data*/"${DBNAME,,}"Backup/"$DBNAME.0.DBPART000".$(date +%Y%m%d)* )
     for BACKUP_FILE in "${BACKUP_FILE_ARRAY[@]}"
             do
             echo "lcd $(dirname $BACKUP_FILE)" >> $FTPFILES1
             echo "put $(basename $BACKUP_FILE)" >> $FTPFILES1
     done
  echo "quit
  end_script
  exit 0  " >> $FTPFILES1

1 Answer 1

2

You misunderstood about the bash. When you run the lftp command you are no longer in a bash but in the lftp environment and there is no CTRL+Z. In your case, I would suggest that you write 6 separate scripts. One for each file. and then write a BASH script that calls lftp individually. Then you can also use &. For example:

lftp -u user,pass 1.1.1.1  << script1 &
lftp -u user,pass 1.1.1.1  << script2 &
...
2
  • then i need to remove exit 0 Commented Jul 5, 2022 at 7:21
  • That depends on your use case. Maybe you have the chance to do some tests. Commented Jul 5, 2022 at 12:02

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .