I want to set disk quota (limit maximum space used in the file system) that a particular process can use under Linux. There seems to be plenty ways to limit disk quota for a user, but not at a per process granularity. One way I can think about is creating a user for each process but as you can imagine that is not a great solution.
Handling it with different user accounts may well be the only possible way since processes do not own any files and can therefore not have a disk quota.
To make it even clearer, at the very best you could manage a quota for the files currently used, should you develop such a kernel patch, but it would still lose its sense to track the files that were written previously and got closed as they are not under its responsibility at all.
Doing such a flawed patch would also result in considerable performance degradation and wouldn't make sense in situations where more than one program opens the same file.
For those and many other reasons, it theoretically simply can not be done properly.
You may be able to use ulimit. It's not exactly a quota, but it's similar.
ulimit -f 1000
will limit the maximum size of any file created by the current shell or any processes invoked by the current shell to 1000 kilobytes each. Like I said, not a quota, because the process can just write a bunch of files under the limit if it wants, but, depending on what you're trying to do, it might serve.
In tcsh the command is limit instead of ulimit and may use slightly different syntax.