1

I'm confused about the actual internal workings of a file system. The question is about file storage vs block storage. From what I understand, the difference is in the level at which we access data: File storage at a file level and block storage at block level.

My question is: Does file storage also ultimately divide the file in to chunks and store it again on individual blocks? And so the difference is really in who is responsible for combining these chunks to form the file? In case of block storage this is done by the OS on that block server and in case of file storage it is the OS with help of the file system?

Follow up: In case of block storage, how does the OS keep track of the various chunks of the same file without using a file system then?

Improve this question
4
  • "who is responsible for combining these chunks to form the file" ... I would have thought that is the job of the filesystem.
    – muru
    Commented Dec 13, 2021 at 5:41
  • hmh, some of the descriptions (e.g. ibm, redhat) I found of block storage are really generic, to the point of being uselessly vague, but if block storage means something that's accessed via e.g. iSCSI or Fibre Channel, as opposed to CIFS or NFS, then the difference would basically be if the filesystem mapping the files to blocks runs on the storage server or on the client using it.
    – ilkkachu
    Commented Dec 13, 2021 at 9:16
  • on file-based storage, the client would send commands like "open file foo.txt, read 1234 bytes from it", vs. block-level commands like "read sectors 34 to 35, then read sectors 120 to 122". In the former case, the client doesn't need to know what the filesystem is, you could run a fileserver with ext4 or ZFS and have a Windows machine access it. In the second case, the server doesn't know or care what the client does with the blocks. Of course, depending on what you do, block storage might not even need a filesystem, e.g. some databases might like to run on top of the raw block device.
    – ilkkachu
    Commented Dec 13, 2021 at 9:18
  • It seems to me you've commingled two different concepts - though they are certainly related. A file system may be defined simply as a system that defines how files are named, stored, and retrieved from a storage device. Block storage generally refers to a non-volatile data storage device (hardware) whose storage units (e.g. sectors) may be abstracted as "blocks" for use by the filesystem.
    – Seamus
    Commented Jan 3, 2022 at 9:38

2 Answers 2

Reset to default
1

A disk driver interfaces with a disk, or disk partition, and presents the system with access to disk blocks 1 through N. The disk driver conceals the dirty details of accessing the underlying "device". Physical disks are accessed by Cylinder / Head / Sector (C/H/S) Changing Cylinders involved physical action, out in the Real World (sliding the arm carrying the read/write Heads in and out). The Heads "flew" over the disk platters. Changing Head took place at electronic speed, but Sector was a matter of waiting for the disk to revolve. Optimizations include sorting disk I/O by C/H/S, minimizing direction changes of the arm, chunking so you read/write ALL the Sectors under a single Head, buffering, ... Details differ among all disks and disk-like devices. Disk drivers make it possible for programs to ignore these details. All disk drives present Blocks 1 to N.

Disks are strictured by way of the GPT or older MBR. Each partition "looks" like a disk (Blocks 1 to N). Wikipedia explains more.

When a partition is "mounted", it is assigned a place in the Filesystem Hierarcy (man hier), and it's filesystem consistency is checked, and if necessary, repaired.

A filesystem allocates these disk blocks(Blocks 1 to N), and keeps track in disk blocks it has allocated to itself (overhead blocks) using i-nodeswhen one runs mkfs to create the filesystem. Read man mkfs fsck. The filesystem organizes blocks into "clusters" and "super clusters" for I/O optimization. All of the I/O (user data, fliesystem overhead) is buffered, and only periodically flushed to disk. The system can crash between the filesystem allocating blocks and the on-disk copy of the filesystem metadata, and become inconsistent. Filesystems use redundancy and journaling to avoid or. recover from inconsistency.

User programs can read/write files, in characters or blocks of (binary) data, according to access rules enforced by the system.

Every layer between the user and the hardware can be optimized (but do you want ALL the memory used for buffers?) without requiring changes in other layers. Blocks 1 to N.

Improve this answer
2
  • ok but the question might also be how one can access directly block storage by-passing the file system? e.g. a database
    – toto'
    Commented Jan 4, 2023 at 14:13
  • 1
    Why would you need to access the actual disk blocks? What's your "use case"? It can be done by umounting all the disk's partitions, and working. with /dev/sdX . Be very careful -- you can trash your disk, confuse the filesystems, or make your disk unmountable. Read man fsck. Backup your data before you start.
    – waltinator
    Commented Feb 21, 2023 at 18:58
-1

For file storage, first there's the superblock that provides high level review of the filesystem:

A superblock is a record of the characteristics of a filesystem, including its size, the block size, the empty and the filled blocks and their respective counts, the size and location of the inode tables, the disk block map and usage information, and the size of the block groups.

But the thing that's "responsible for combining these chunks to form the file" is the inode - which is managed by the filesystem itself, is usually located in the inode table (or other similar structure of some sort), and contains the metadata of the file, including the disk block locations of the object's data.

For block storage, what "keeps track of the various chunks of the same file" depends on the implementation. There could be some software responsible for managing the blocks, usually in the form of a database, although it is also possible that the block storage server just stores the blocks on some filesystem itself (or both).

Improve this answer
2
  • The superblock is just a data structure that describes some things of the filesystem in whole, as opposed to describing some smaller part of the filesystem, a file/inode/directory, or even a block group for ones that have those. The superblock doesn't "do" anything, it's the filesystem code that does.
    – ilkkachu
    Commented Dec 13, 2021 at 8:50
  • @ilkkachu you're right, I sent this answer too early before I wrote everything I intended to, and then just forgot about it. I've updated my answer.
    – aviro
    Commented Dec 13, 2021 at 9:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .