I was wondering if I have a file that has apparent size of 1 GiB but actual size of 0 B can get copied to, for example, a USB flash stick with 512 kiB free space?

You can create a file as such using:

dd if=/dev/null of=big-file bs=1 seek=1GiB

Now you can see the apparent and actual sizes:

du --apparent-size -hs big-file

Note: remove the --apparent-size option to get the actual size.

So, the main question is: which size is taken into consideration when copying a file to some directory, be that a hard disk drive, USB flash stick, DVD, etc?

  • 2
    It depends on if filesystem supports fallocate or not. – Ipor Sircer Oct 3 '18 at 14:55
  • 4
    That depends very much on the filesystem of the target device, and if it supports sparse files. – dirkt Oct 3 '18 at 14:55

Actual size.

Of the destination file.

The simplest implementation is to not try to predict this in advance.

In addition to that, a small fraction is required as overhead. This overhead would be even more complex to predict.

Support for sparse files varies, depending on both the filesystem, and the program making the copy. Most notably, FAT filesystems do not support sparse files.

GNU cp automatically detects and handles sparse files. It does not check for space in advance. (Based on the documentation for GNU coreutils 8.29)

By default, sparse SOURCE files are detected by a crude heuristic and the corresponding DEST file is made sparse as well. That is the behavior selected by --sparse=auto. Specify --sparse=always to create a sparse DEST file whenever the SOURCE file contains a long enough sequence of zero bytes.

Notice that filesystems can have different block sizes. This can cause the amount of space used by the sparse file to increase, or even decrease!

( Writing file data generally also requires internal filesystem structures which take up space of their own. This is not shown in the file's size - neither the actual nor the apparent size. In traditional UNIX filesystem design, you also need a free inode. The space available for inodes is shown by df -i. And most obviously, you need some space to store the file name :-) ; this is often shown in the size of the parent directory. This is discussed specifically in the question Can I run out of disk space by creating a very large number of empty files? )

  • "Support for sparse files varies, depending on both the filesystem, and the program making the copy." -- This could be in bold as the first thing here... And since we're talking USB sticks, chances are it's formatted as FAT32, which I don't think will support sparse files. – ilkkachu Oct 3 '18 at 17:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.