Open file from particular position?

Question

I was wondering, if you had a particularly large file, saying 64 MB, is it possible to figure out the physical hard drive location of the file, and then read bytes from a particular offset into the file?

Let's say I'm interested in 100 bytes with an offset of 60 MB from the start of the file. I don't want the inefficiencies of the hundreds of disk seeks it would take to get from the start of the file to the end of the file if I used some application level seek() function.

Is there a solution?

Thanks so much!

Answer 1

You seem to have a misconception as to how seek() behaves. It does locate the place where the data at this offset is stored as efficiently as possible, without reading the intervening bytes. There will be a few seeks (likely not hundreds), to traverse the block index.

What you can't do is save the traversal of the block index from one time when the file is opened to the next. The OS would have to remember that the file hasn't been modified or relocated since the last time it was openeed, which would require memorizing a lot of data for a very small potential gain.

Note that the contents of a file are not in consecutive disk positions, in general. Files tend to be fragmented. Filesystems usually try to reduce fragmentation, but this cannot be guaranteed in general.

Answer 2

Rereading, it seems like maybe I didn't answer the underlying question:

Using "seek" at the application (really, kernel) level doesn't necessarily cost any "seeks" on disk - all it does is update the offset number associated with the file handle.

Once you ask the kernel to read or write it will translate that offset into a disk offset, which may involve reading blocks to figure that out, but has a best case cost of one seek - just like your direct access would.

It is absolutely possible to do that: that is exactly what the file system driver does, after all, so it must be possible for someone else. All you need is access to the raw disk.

There are of examples of people doing that for existing file system formats. You can also do this by hand if you want.

If the file system is in active use you have some technical challenges that make it harder to do - because the content on disk is changing in a way that you can't quite see - but it is still possible.

You can also ask the kernel directly; the xfs_bmap tool does that, and at least some file systems implement the same interface so you can directly ask.

Calculating the location will take the same number of seeks that the kernel would take, though, so you are unlikely to actually save anything doing this.

Answer 3

I don't think so.

If you open the file, you'll either be at the start (for read/write) or the end (for append). Even in "update mode" you won't simply land in some specified location in the middle of the file.

I think the best you can do is what you already eluded to: If you can calculate the offset from the start you could seek to that location directly and read the data. I don't think this would involve any excessive read operations in-between. You next read after opening the file should be at the calculated offset.