I have a folder(which contains a lot of sub-folders and files) on a machine, I used

   du -m 

and it shows the disk usage of all sub-folders and files, anyway, the overall disk usage is 78M

I used scp -r to copy the folder into another machine, this time, du -m get the overall disk usage: 12M, very different.

Why does this happen?

I'm afraid some of the files or sub-folders are not copied fully, so are there any other ways to check the total number of bytes?

  • 1
    Were there errors during the copy? Are all the files present in the copy? Do they have the same size? What was the exact scp command line: were you just copying the directory? – Gilles May 7 '13 at 1:06
  • there is no errors. I think all files are present in the copy. They don't have the same size, but they have the same content. I used scp -r – misteryes May 7 '13 at 8:25
  • Do you really have all the files (compare the output of find | sort)? If not, what can you spot about the ones that are missing? If you do, do all the files have the same size (compare the output of du -ak | sort -k2)? – Gilles May 7 '13 at 8:28
  • I used du -k|wc -l, the number of files/subfolders is the same. But for the same file, the size is different, e.g, for a file on one machine it is 4K with du -k but 16 bytes with du -b, on another machine it is 0K with du -k and 16 bytes with du -b – misteryes May 7 '13 at 9:30
  • What are the operating systems (Linux/OSX/Solaris/FreeBSD/Windows/…) and filesystems (ext4/btrfs/hpfs+/ffs/ntfs/…) on both sides? Are these a few big files or are there a lot of small files — how many files are there in total? – Gilles May 7 '13 at 10:04

Several possibilities.

a) scp may not have copied hidden files/directories; hard to tell w/o knowing your data, seeing how you invoked scp.

b) another scenario will be that the filesystems use different block-sizes, and du doesn't add up the files byte sizes, it measures occupied disk space.

  • so how to get the file byte sizes? – misteryes May 6 '13 at 21:39
  • and how to use scp so that even hidden files can be copied, but anyway, I don't think there are hidden files because the folder is full of data files which are produced by some of my programs – misteryes May 6 '13 at 21:40
  • Have you considered using rsync? As for the how .. show us how you did it, and describe the directory layout. – tink May 6 '13 at 21:49
  • @misteryes scp doesn't care about dot files ("hidden files") so don't worry. Just the shell does when you invoke scp: scp -r *.foo ... – Hauke Laging May 6 '13 at 22:08
  • what is Just the shell does when you invoke scp: scp -r *.foo ...? – misteryes May 6 '13 at 22:14

Another possibility, which is remote in my judgement: your original directory and the directories residing inside it have had a lot of files added and deleted. In most filesystems, directories only grow in size, never shrink. The missing space could constitute unused slots in directories.

  • are there any other ways to check the total number of bytes? – misteryes May 6 '13 at 22:15
  • But you can see the directory size, too: ls -ld dir – Hauke Laging May 6 '13 at 22:27
  • no, it is not correct, it shows 47 in the second field on both machines, but du -m --summarize show 12 on one machine and 89 on another machine – misteryes May 6 '13 at 23:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.