Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Actually IOzone is a filesystem benchmark tool. The benchmark generates and measures a variety of file operations like sequential reading/writing, random reading/writing. So I want to customize my own Linux file system ext3 either for random reading(Database intensive or OLTP) or sequential reading(audio/video app) so that there will be better performance for specific application. And how can we change file systems parameters like block size, disk cache, buffer size etc to achieve better performance. I am newbie in Linux file systems. So please tell me, where will I get excellent sources for reading about this task and load again this modified file system as a modules or in another way.

Thanks

share|improve this question

1 Answer 1

To change these things you either have to do them on filesystem creation with mke2fs ( which mkfs.ext4 for example uses ) or some of the tuning options are available after creation with with tune2fs. This Forum thread on Gentoo Forums was about ext3 tuning. Here's what I think are the important parts from that ( it's all copied with some formatting ) *( Also please be advised that data=journal disables some certain peformance enhancements in ext4)*

Using Directory Indexing

This feature improves file access in large directories or directories containing many files by using hashed binary trees to store the directory information. It's perfectly safe to use, and it provides a fairly substantial improvement in most cases; so it's a good idea to enable it:

tune2fs -O dir_index /dev/hdXY

This will only take effect with directories created on that filesystem after tune2fs is run. In order to apply this to currently existing directories, we must run the e2fsck utility to optimize and reindex the directories on the filesystem:

e2fsck -D /dev/hdXY 

Enable Full Journaling

By default, ext3 partitions mount with the 'ordered' data mode. In this mode, all data is written to the main filesystem and its metadata is committed to the journal, whose blocks are logically grouped into transactions to decrease disk I/O. This tends to be a good default for most people. However, I've found a method that increases both reliability and performance (in some situations): journaling everything, including the file data itself (known as 'journal' data mode). Normally, one would think that journaling all data would decrease performance, because the data is written to disk twice: once to the journal then later committed to the main filesystem, but this does not seem to be the case. I've enabled it on all nine of my partitions and have only seen a minor performance loss in deleting large files. In fact, doing this can actually improve performance on a filesystem where much reading and writing is to be done simultaneously. See this article written by Daniel Robbins on IBM's website for more information

In fact, putting /usr/portage on its own ext3 partition with journal data mode seems to have decreased the time it takes to run emerge --sync significantly. I've also seen slight improvements in compile time.

There are two different ways to activate journal data mode. The first is by adding data=journal as a mount option in /etc/fstab. If you do it this way and want your root filesystem to also use it, you should also pass rootflags=data=journal as a kernel parameter in your bootloader's configuration. In the second method, you will use tune2fs to modify the default mount options in the filesystem's superblock:

tune2fs -O has_journal -o journal_data /dev/hdXY

Please note that the second method may not work for older kernels. Especially Linux 2.4.20 and below will likely disregard the default mount options on the superblock. If you're feeling adventurous you may also want to tweak the journal size. (I've left the journal size at the default.) A larger journal may give you better performance (at the cost of more disk space and longer recovery times). Please be sure to read the relevant section of the tune2fs manual before doing so:

tune2fs -J size=$SIZE /dev/hdXY

Checking The Filesystem Options Using tune2fs

Well, now that we've tweaked our filesystem, we want to make sure those tweaks are applied, right? Surprisingly, we can do this options iusing the tune2fs utility quite easily. To list all the contents of the filesystem's superblock, we can pass the "-l" (lowercase "L") option to tune2fs:

tune2fs -l /dev/hdXY

Unlike the other tune2fs calls, this can be run on a mounted filesystem without harm, since it doesn't access or attempt to change the filesystem at such a low level.

This will give you a lot of information about the filesystem, including the block/inode information, as well as the filesystem features and default mount options, which we are looking for. If all goes well, the relevant part of the output should include "dir_index" and "has_journal" flags in the Filesystem features listing, and should show a default mount option of "journal_data".

share|improve this answer
    
tune2fs and mke2fs provide options like buffer size, error checking, journaling etc. These options are quite useful but how can I tune file system for sequential/random reading/writing. –  Sushant Jain Apr 9 '11 at 4:47
    
@Sushant reread, and read the linked IBM article. This is the extent of my knowledge. Unfortunately what I know suggestions that improving read performance in this way hurts write performance to an extent. using modes like writeback will improve write performance, but will not be as likely to server a hard reboot. ext4 has many tuneable options that will affect write performance. Also benchmark, benchmark, benchmark. Your hardware affects a lot. –  xenoterracide Apr 9 '11 at 4:58
    
Actually I am not talking specifically about benchmarking of filesystems. I need to know about ways to customize file system for specific purpose. And I have get a linux command "hdparm" for changing hardware parameters. I wish it will be helpful for me. –  Sushant Jain Apr 9 '11 at 5:35
    
@sushant well I hope what I've given you so far is helpful. Maybe someone else will know more, as I've already suggested several things that do what you're asking, and I'm not sure what more you're asking for. My Ext4 filesystems are optimized for durability and reads, but they're slower than they could be for writes. If you think you're going to get optimized for all of these "sequential/random reading/writing" use the defaults, that's what they're there for is a good balance. –  xenoterracide Apr 9 '11 at 7:48
    
@sushant I'll be honest, I'm not sure what you think you can do with these filesystems that I haven't suggested... sometimes you don't need to tune the filesystem, but choose a filesystem better suited to the job, e.g. reiserfs is infinitely better at handling a large number of really small files. –  xenoterracide Apr 9 '11 at 8:08

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.