Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Is it due to the buffering in memory? I did cp /network/large_file . & and I kept doing ls -lh to monitor progress, but the file didn't appear until it was about 400MB. Linux 2.6 kernel, Bash 4, 4GB swap, 16GB RAM.

share|improve this question
    
You'll probably get better answers if you tell us how you're copying this file from the network. –  rahmu Aug 30 '13 at 22:19
    
The file is on a CIFS share, mounted with mount.cifs to /mnt/network being copied to an EXT4 filesystem using the commands I listed. What else would you like to know? The file is ~ 260 GB. –  Gregg Leventhal Aug 31 '13 at 1:49
    
Also The local hard disk is virtualized in VMware (vmdk format) and the backing datastore is a hardware RAID 5 with 5 disks using a PERC controller in a Dell R620 server. –  Gregg Leventhal Aug 31 '13 at 1:58

1 Answer 1

This is likely to be dependent on your underlying filesystem, possible hdd buffers, raid controller buffers (if you are using raid), etc. When a file is copied, it first starts the stream in memory, notifying the disk there is a pending write operation - and the disk takes care of it from there. It's very common for a write to have "completed", but the file is actually still being written by the disk. Some filesystems prevent this "lag" such as ZFS and I believe BtrFS (this is known as copy-on-write). Other filesystems will report the copy/write operation is completed when it's not really -- which can lead to file corruption if power is lost, etc. ZFS and BtrFS don't report back to the OS that a file write operation is completed until it is fully complete -- preventing this situation.

To test this, try doing a copy of something largerish - such as a DVD iso of your favorite distro from one directory to another. The command line will return to the prompt in a short while, but if you immediately type sync and send the command, you'll notice it will hang the command line until the write operation is fully complete. (if you have disk activity lights, you'll also notice they are still blinking)

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.