Tell me more ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.
up vote 4 down vote favorite

Is there a tool that monitors a certain directory and if a program tries to access a file in that directory, the program loads that file from another place transparent for any application that tries to load that file?

The background is that I have a local development enviroment for a large server application. At the moment I sync all the files created by the users on the production system from time to time via rsync to my local machine. But the amount of data has become too big. I can not store the several hundreds of gigabytes from the production environment on my local dev machine. So is there a way of mimicking the files from the external system and load/sync them only from the external system if they are really needed?

Writing changes to these "monitored" directories should nevertheless take only place locally.

share|improve this question

3 Answers

up vote 2 down vote accepted

Your situation sounds like a natural use for unionfs. With unionfs you could mount the production system's data as a read-only filesystem and mount a transparent writable filesystem layer on top of it that would accept your local changes.

share|improve this answer
This sounds pretty interesting, I didn't know about that. Will devinitly give it a try. Thanks. – marcusx Jun 18 '12 at 8:52
up vote 2 down vote

Maybe it would be good solution to mount remote filesystem over network. There are many possibilities to achieve that.

You can export some directories from production machine over NFS, SMB or SSHFS and then mount it on your dev machine. If you use this method files will be transfered over network only if you request it.

I don't know what is a purpose to transfer users files from production system to dev system, but you should remember about security of this solution if files contains some private/sensitive data. If you choose option to mount remote file system remember to mount it in read only mode to avoid accidental destruction.

share|improve this answer
It's mainly image and print data as pdf's. And the reason to have it local is that I need to display the images in my local browser and that I generate PDF's with the image data. So you cannot work or especially hunt bugs if you have no data locally. But anyway - just mounting a remote directory will not do it. I need do be able to operate on the files. But as you said it is not an option to write back to the productive (or staging or whatever) system (which holds the complete dataset) from local. – marcusx Jun 5 '12 at 16:46
up vote 1 down vote

Looks like a job for FS-Cache. This is a Linux filesystem that interfaces between applications and the real filesystem (which can be NFS or SSHfs if you're caching remote files).

The quickstart guide (from the Gentoo Wiki):

cachefilesd
mount -t nfs -o fsc remotehost:/somedir /mount/point
ls /var/cache/fscache

For more information, see The Fedora documentation and Bart's Blog.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.