Am I correct that chunk size in context of RAID is essentially the same thing as cluster in file-system context? In other words, chunk size is the smallest unit of data which can be written to a member of RAID array? For example if I have a chunk size of 64KiB and I need to write a 4KiB file and cluster size of the file-system is also 4KiB, then is it true that I will use one 64KiB chunk and basically waste 60KiB?
Add a comment
|
2 Answers
Given that chunks can be quite big and that the parity information is simple XOR (i.e. does not affect data before or after the piece in question) the assumption that only complete chunks can be written does not make sense to me.
Chunks are the unit in which data is spread over the volumes. One chunk of continuous data is written to a certain volume, the next data is written to another.
Both with file systems and with RAID this is an optimization issue: In a file system too small blocks / clusters would cause metadata overhead, too big blocks waste too much space (as most file systems can use a certain block for a single file only).
With RAID it is similar: If you have tiny chunks then you need accesses to several disks even for very small files (or other data). In most cases the higher latency of the (in this single case) slower drive takes more time than reading from one drive alone. This is not valid for SSDs but they are not the dominant technology for RAID.
If you have very big chunks then even those accesses which could be clearly speeded up by spreading to several drives are done to only one drive.
answered Mar 5, 2014 at 20:55
The answer to the OP's question is: Yes. In a RAID, a "chunk" is the minimum amount of data read or written to each data disk in the array during a single read/write operation.
In your example, you won't necessarily "waste" 60-KiB as you put it. That depends on the combination of the file system structure and the underlying RAID structure. However, you are raising a very important point, which is that it's ideal if the file system configuration aligns with the RAID configuration with regard to their units of storage.
Continuing with your hypothetical example; if you had a RAID with 64-KiB chunk size and an overlaid file system using 64-KiB block sizes, then yes a 4-KiB file would use an entire 64-KiB area of storage space in the file system all by itself. And at the same time it would eat up an entire 64-KiB chunk in the RAID all by itself. However, that would be because the file system was setup to 64-KiB blocks. Those blocks are the smallest unit of storage data of that file system. Any file size smaller than that block will still use 1 block of file system storage space.
My point is your 2nd question is actually relevant to the file system and not the RAID.
Continuing with my example above, if your RAID used 16-KiB chunks and your file system on top of the RAID used 64-KiB blocks, then each block written to the RAID would require 4 of those chunks (64/16=4).
Now reverse that thought process. What if you had 64-KiB RAID chunks and 16-KiB file system blocks? Now each file system block only uses 1/4 of the RAID chunk. That means 1) your 4-KiB file takes up 16-KiB in the file system; and 2) the RAID will perform a read/modify/write operation when writing that 4-KiB file/16-KiB block because the RAID's smallest unit of storage is 64-KiB. So then your file system is more efficient, but your RAID is less efficient (for that particular file operation).
