The behaviour you are observing happens because even though the client has written data to the socket for the FTP data connection and the client's kernel has replied that this happened successfully (by returning successfully from the
send() system call), the data is still in all sort of buffers and not already committed to the destination file. It's in network buffers in the source OS, it's in buffers in ethernet cards, it's in flight on the wire, it's in buffers at the destination, it's in the FTP server process' buffers, it's been written to a file on the server but not flushed to disk, etc...
It doesn't only happen with FTP. You can easily notice this happening with more modern file transfer tools like
scp a file halfway aqround the planet (for a high-latency network) or to a system with slow storage, and you will see that the transfer progress is shown as faster than it really is, then it seems to hang at 100% complete for a while before the command actually completes.
If you look at the source code for netkit-ftp (the default/standard FTP client on Debian), its # mark processing does not do anything special. It writes the hash marks according to how much data it has written to the data connection socket. Thus, it will exhibit the same behaviour you are observing.
What normal FTP clients do, and what you should do too, is wait for the reply to the
STOR command on the control connection. This reply will come some time after you have already flushed and closed the data connection socket. When you get that reply, you know that the FTP server has the data. It might not be fully committed to disk over there, but that's as much of a confirmation as you're going to get.