4

I have a folder(which contains a lot of sub-folders and files) on a machine, I used

   du -m 

and it shows the disk usage of all sub-folders and files, anyway, the overall disk usage is 78M

I used scp -r to copy the folder into another machine, this time, du -m get the overall disk usage: 12M, very different.

Why does this happen?

I'm afraid some of the files or sub-folders are not copied fully, so are there any other ways to check the total number of bytes?

6
  • 1
    Were there errors during the copy? Are all the files present in the copy? Do they have the same size? What was the exact scp command line: were you just copying the directory? May 7, 2013 at 1:06
  • there is no errors. I think all files are present in the copy. They don't have the same size, but they have the same content. I used scp -r
    – misteryes
    May 7, 2013 at 8:25
  • Do you really have all the files (compare the output of find | sort)? If not, what can you spot about the ones that are missing? If you do, do all the files have the same size (compare the output of du -ak | sort -k2)? May 7, 2013 at 8:28
  • I used du -k|wc -l, the number of files/subfolders is the same. But for the same file, the size is different, e.g, for a file on one machine it is 4K with du -k but 16 bytes with du -b, on another machine it is 0K with du -k and 16 bytes with du -b
    – misteryes
    May 7, 2013 at 9:30
  • What are the operating systems (Linux/OSX/Solaris/FreeBSD/Windows/…) and filesystems (ext4/btrfs/hpfs+/ffs/ntfs/…) on both sides? Are these a few big files or are there a lot of small files — how many files are there in total? May 7, 2013 at 10:04

2 Answers 2

3

Several possibilities.

a) scp may not have copied hidden files/directories; hard to tell w/o knowing your data, seeing how you invoked scp.

b) another scenario will be that the filesystems use different block-sizes, and du doesn't add up the files byte sizes, it measures occupied disk space.

7
  • so how to get the file byte sizes?
    – misteryes
    May 6, 2013 at 21:39
  • and how to use scp so that even hidden files can be copied, but anyway, I don't think there are hidden files because the folder is full of data files which are produced by some of my programs
    – misteryes
    May 6, 2013 at 21:40
  • Have you considered using rsync? As for the how .. show us how you did it, and describe the directory layout.
    – tink
    May 6, 2013 at 21:49
  • @misteryes scp doesn't care about dot files ("hidden files") so don't worry. Just the shell does when you invoke scp: scp -r *.foo ... May 6, 2013 at 22:08
  • what is Just the shell does when you invoke scp: scp -r *.foo ...?
    – misteryes
    May 6, 2013 at 22:14
2

Another possibility, which is remote in my judgement: your original directory and the directories residing inside it have had a lot of files added and deleted. In most filesystems, directories only grow in size, never shrink. The missing space could constitute unused slots in directories.

3
  • are there any other ways to check the total number of bytes?
    – misteryes
    May 6, 2013 at 22:15
  • But you can see the directory size, too: ls -ld dir May 6, 2013 at 22:27
  • no, it is not correct, it shows 47 in the second field on both machines, but du -m --summarize show 12 on one machine and 89 on another machine
    – misteryes
    May 6, 2013 at 23:13

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .