I need to download some data from some SFTP connection and delete if on that FTP afterwards. But I have to be sure to have downloaded the data correctly.
At the moment I just do this in my script run by a cron:
sftp -oIdentityFile=key user@server <<EOF mget -r * mdelete * bye EOF
But I worry that if a download breaks because of any reason the files get deleted anyway, since the FTP connection just works the next command without breaking the connection after a failed action.
I need to be sure the files have been downloaded correctly, best would be a hash compare. But since I heavily doubt FTP offers that function, what would be the best solution to solve that problem?
IF NOT possible: It might be a workaround to leave all the files on the FTP server and just download the new ones. This seems to be a way easier solution. Any advice for this solution?
EDIT: Since I have no influence on the server whatsoever, I am searching for the most practicable solution to be sure to download all files before they get deleted. I am not so much worried about the integrity of the data itself (I do trust ssh there).