I am trying to understand the differences between these three file system at a very basic level.

  • Distributed FS: HDFS
  • Parallel FS : Lustre
  • Traditional FS : ext4/ext3/ NTFS/FAT etc.

I want to know what are the basic conceptual differences between these three file system. Most of my knowledge is of the traditional file systems , i.e. ext3/4 superblock , inode etc.

  • If a MPI based process(np=8) tries to read a file or write a file A from file system, then how does the file access mechanism differ in these contexts
  • also how is a file stored in this environment? i.e. File A will be split across multiple disks or file A will have redundant copies on storage. or a more simple scenario will be say multiple users opens a word document then saves it, then how does the write-back/synchronization differ in these 3 scenarios

So far I have formed a few concepts that:-

  • In local file system , the storage is physically mounted on server/nodes.
  • In parallel file system , a disk is shared (mount) on multiple nodes, and,
  • In distributed FS, the multiple nodes have multiple local storage but all of them are synchronized by some mechanism.

If I have A,B are a workstation and C,D is the disk:

  1. If C is physically mounted on A & formatted as ext4 then it is traditional file system.
  2. If C is physically mounted on storage server Z + C is network mounted (NFS) on both A & B then this is cluster FS.
  3. If C is physically mounted on A and network mounted on B, D is physically on B and network mounted on A. Then this gives rise to Distributed FS.

Though some answers state that metadata and data are on separate servers in parallel file systems , but here too I wish to understand how metadata is managed in Distributed File Systems?

  • So far i came to know that The traditional file systems cannot be exported , where as the lustre & hdfs can be exported . Also the files will have redundant copies in DFS, where as the data is striped (and duplicated) in PFS. i.e. if we request a file then one node out will dispatch the file in DFS where as in PFS , multiple nodes will be involved for file transfer.
    – puneet336
    Jul 21, 2015 at 6:51

2 Answers 2


AFAICS, the term "parallel filesystem" is marketing B.S. that just means that the filesystem driver was built with the understanding that multiple processes can write to files at the same time, and so uses an appropriate block allocation strategy to write the files contiguously to different parts of the disk, instead of fragmenting them on top of each other. This has pretty much been standard practice on unix systems for 20 years.

A clustered filesystem is one designed to be stored on a SAN, where the "disk" ( which may actually be a raid array implemented in the hardware of the SAN disk box ) is directly accessible to multiple hosts ( over a shared SCSI connection ), and so can be simultaneously mounted by multiple hosts at the same time, as the filesystem drivers take special care not to step on each other. This is entirely different from sharing a regular filesystem over the network with NFS or CIFS.

A distributed filesystem is more like a network filesystem, but it internally communicates with multiple servers to distribute the load across them, in a way that is largely transparent to the user. An example is afs.


The difference between NAS and SAN is being removed. There are discussions going on. I am on the side of there is no difference between them.

How you mount those are not different anymore. Focusing on functional benefit is more understandable by asking questions such as traditional file system; is it distributed or clustered? You may ask the same questions for distributed and clustered filesystems and there is distributed clustered filesystems.

Question of what makes a filesystem distributed or clustered lets you get rid of short-name confusion. What you expect in a clustered systems is session sharing and synchronization. This is not the case what I get from the current clustered filesystems they are more like asynchronous because of CAP the are restrictions.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.