Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

If I understand it correctly, "fsck" is used in UNIX systems to check for internal consistency within disks after there has been a crash. I was wondering, then, how it uses "metadata" to make sure that everything in the filesystem is consistent, for instance what steps the fsck process needs to take in order to restore and repair?

-> edit: it would also be helpful if someone could clarify what "synchronous write-through" refers to in this picture as well.

share|improve this question
    
This seems like it would be more on-topic on U&L, maybe. –  Aaronaught Oct 25 '11 at 2:18
add comment

migrated from programmers.stackexchange.com Oct 25 '11 at 3:11

This question came from our site for professional programmers interested in conceptual questions about software development.

3 Answers

up vote 6 down vote accepted

Metadata is information about data. If you think about a text file, the sequence of letters that text file is the actual data the file contains. However, the file has a name, an owner, a creation date, a location on the the storage medium, etc. All this information is metadata. Note that metadata can be handled the same way that data is. For instance, Unix stores file metadata in data files called directories (or folders in this new age of computing :-)

The Unix file system uses a basic unit of storage called an inodes. An individual inode can either contain actual file data, directory information (metadata), or be unused (free). Note that the act of creating a new file involves changing the state of an inode from free to allocated, writing data to the new file, and writing metadata to a directory file. It is possible for a computer to crash in the middle of this type of operation, in which case the file system can be corrupted.

File system checking consists of reading all the inodes and attempting to resolve as many corruption issues as possible. For instance, suppose an inode is not on the list of free inodes, but there are no directory entries that say that this inode is part of a file in any of the directories that the file system knows about. This inode can be placed back on the list of free inodes.

Synchronous write-thru is a way of performing writing to the disk in a manner that assures that if a crash does occur, the file system can be recovered. For instance, when you are creating a new file, you need to allocate an inode, create the inode with its metadata set, then update the file containing the directory information. With synchronous write-thru, these are done as separate actions, one at a time, in that order. If the crash occurs before the directory is written, then the inode can be placed back on the free list, and the file creation did not occur.

Other types of file system checks are possible as well.

share|improve this answer
add comment

In the context of Unix or Linux file systems, "metadata" is information about a file: user ID of who owns it, permissions, file type (special, regular, named pipe, etc) and which disk-blocks the file uses. That's all typically kept in an on-disk structure called an "inode". One of the pieces of information in an inode is how many "links" to the file exist. A regular file typically has 1, but a directory (which is mostly a file with a special "I'm a directory" file type mark) has at least 2 links. Everything visible in a file system has one link from the directory in which it appears, but since every directory has a "." name as well as its ordinary name, they have 2 links.

fsck can scan a file system's "inode blocks" (the disk-blocks that contain the inode data structures) to find inodes that have a link count greater than 0. A file represented by an inode with a link count greater than 0 should appear in a directory somewhere. If that inode does not appear in a directory, fsck puts the file in well-known directory, usually "lost+found" at the top of the file system. Note that Unix/Linux file systems typically don't have any metadata in an inode about which directory the inode belongs to, only information in the directory file about which files the directory contains.

fsck can use other metadata like which disk-blocks contains a file's data. fsck can check if disk-blocks that an inode says belong to a file appear in the file system's "unallocated list" of disk-blocks. Potentially fsck could check if two or more inodes contain the same disk-blocks, which would indicate some kind of multiple-allocation corruption.

The allocation of disk-blocks to files appears in the on-disk inodes. The directory membership tree-structure of a file system appears in the directories. The inodes don't appear, and aren't allocated to, directories. fsck takes advantage of this seperation to do repairs. This is quite different from systems like MS-DOS or early Windows, where a "file allocation table" held both tree-structure (directory membership) and disk-block allocation. Corrupt the "FAT" and you've got to go scan disk blocks to see what they contain, and how they might fit together.

share|improve this answer
add comment

Mata data is "Data about Data".

In the case of the *nix file system the meta data used by "fsck" from a header on each data block which points back to a directory entry or the next block of data ,and, the corresponding entries in the directory. "fsck" scans the blocks and checks that the directory entries match and that that forward pointers to the next block are correct.

This varies greatly between file systems. In fact what meta data is stored and where it is stored is what most differentiates the various *nix file systems. The more modern ones also keep a log file of changes to the directory structures.

share|improve this answer
    
Just curious, but do you know how or where modern implementations store the metadata on disk? –  Kaitlyn Mcmordie Oct 25 '11 at 2:08
1  
@KaitlynMcmordie, depends on the fs. For ext[234], most of the metadata is stored in the file's inode. The name(s) is/are stored in the directories, which is to say the data part of files that have the directory flag and a specific format for their main data. File data is stored in data blocks, and inodes are stored in the inode tables, which are allocated when the fs is formatted. Other data in the inode includes the owner, permissions, access timestamps, and pointers to the data blocks. –  psusi Oct 25 '11 at 3:30
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.