0

I want to create a copy the RAM used by a certain program to a file. And then restore that state again later.

Something similar to the effect of ctrl+z & fg, but I also want to free the RAM from that program.

  • For Linux, take a look at criu.org (I personally never got it working spotlessly) – A.B Apr 22 '18 at 21:35
  • 1
    Please edit your question to explain if you want to restore the program state after a long time (including reboot) and what kind of programs and applications (HPC?) do you have in mind. Can you improve the code of that program? – Basile Starynkevitch Apr 24 '18 at 4:54
  • 1
    use a lxc container to run your app and suspend the container when needed – Kiwy Apr 24 '18 at 9:43
2

I want to create a copy the RAM used by a certain program to a file. And then restore that state again later.

Misconception: a process has its virtual address space and uses virtual memory. The RAM itself is a resource managed by the OS kernel. Read Operating System: Three Easy Pieces (a process does not use directly RAM).

(you'll better edit and improve your question a lot to explain much more about it: what kind of program do you have in mind, in what context, why, for how long do you need to "stop" the program, do you need to restart later an improved version of it...? without these details we cannot help much more.)

On Linux, you could use proc(5) to query the virtual address space of some given process. Try cat /proc/$$/maps and cat /proc/self/maps

I also want to free the RAM

No need for that, since the kernel is managing the RAM (sometimes thrashing could happen). See also madvise(2), posix_fadvise(2), mmap(2), mlock(2). When a process terminates, the kernel will release its virtual address space, and later reuses RAM allocated for it. When a process stops (e.g. thru Ctrl Z sending SIGTSTP, see signal(7) & termios(3)), the kernel might reuse its RAM for other purposes (and use swap space to store dirty pages -i.e. page out- of that stopped process). Read about demand paging & http://linuxatemyram.com/

What you want is related to application checkpointing and orthogonal persistence. On Unix and Linux (and most other OSes, including Windows, Android, MacOSX, ...) it is not possible or very difficult in general (how would you handle opened file descriptors, child processes, sockets, ASLR, semaphores, threads, file locking, graphical user interfaces, shared libraries, etc...). But you could write an application with such feature (and you can find libraries helping in that); of course you'll follow some additional conventions and restrictions to make persistence or checkpointing feasible and practical.

If you want that system-wide, consider hibernation.

Persistence is something to be thought of very early at design time of your application (it may be difficult to add afterwards). Notice that databases (sqlite, RDBMS, nosql databases, ...) and indexed files (gdbm ...) can be viewed as a common way to achieve some kind of persistence (you could view your heap as a cyclic graph of objects). Persisting code-related data (e.g. classes, vtables, closures, function pointers ...) is hard in general.

You can find some libraries for checkpointing, e.g. BLCR or CRIU. Of course they work in a limited context on applications developed to use them.

At last, from the algorithmic point of view, persisting the entire state (or checkpointing it) is very close to copying precise garbage collectors. So reading something about them, e.g. the GC handbook, is useful.

However, genuine persistence or checkpointing is difficult to implement and should be though of early at design time of your application. In many cases, it is hard enough to require a full rewrite of an application not providing it.

Stay compatible with the evolution of the code is even harder (e.g. being able to restart with a newer version of your code applied to an old checkpoint). You might get inspired by dynamic software updating techniques.

Some programming languages implementations (e.g. Ocaml, Python, Java, ...) provide serialization or marshalling facilities that could help. Others have some way of checkpointing (e.g. SBCL save-lisp-and-die, PolyML export). Homoiconicity and reflection are helpful programming language features.

  • Thanks for the technical info. Would like to know more about the libs / syscalls which would help achieve that – Omar Elrefaei Apr 24 '18 at 1:49
  • It is not that easy. I added more references. – Basile Starynkevitch Apr 24 '18 at 4:46
4

Something similar to the effect of ctrl+z & fg, but I also want to free the RAM from that program.

Ctrl-z and fg will do exactly what you want: When the kernel needs RAM, it will write out the RAM the suspended program uses to disk into the swap partition, so it can re-use the RAM for something else. When the program is activated with fg later, it will read in that RAM from swap.

  • Won'tr work on the long term. You cannot persist a process and restart it after reboot that way. – Basile Starynkevitch Apr 22 '18 at 17:00
  • 3
    @BasileStarynkevitch: The question said nothing about "it needs to persist after a reboot". The phrasing "but I also want to free the RAM" suggests that surviving reboots is probably not the intention in the first place, and the poster just didn't realize virtual memory will already take care of saving the RAM. – dirkt Apr 22 '18 at 17:10
1

Other than Ctrl+Z & fg it's not likely that you will be able to do what you want.

The problem is that while it's possible to save the memory of a process to a file (for example producing a core-dump) you seem to aim at being able to remove the process from the kernel's process table. But when doing that you will release all other OS resources hold by the process, for example open files etc.

Normally when creating those resources the method of their acquisition is not stored - which means that there is nowhere stored information on how to reacquisite the resources. Even if that information was saved somewhere there's no guarantee that reacqusite them later is possible anyway (the successful acquisition may depend on state external to the process or even computer which might have changed).

0

Possibly of some interest, early versions of the document generation program TeX and LaTeX built their variants using tools which would start the base program, load all the various modules required, then trigger a core dump, and build a new "pre-configured" executable from the core dump.

There's some good discussion over on tex.stackexchange.

I think even in the day it was considered an odd way to proceed, and it only really effected a form of run-time pre-linking, it didn't preserve the state of the running process (dynamic variable values, etc.) at all.

Apparently early versions of emacs did this also, I had not known that.

  • Current TeX (or emacs) has some persistence, but does not trigger a core dump for that – Basile Starynkevitch Apr 24 '18 at 11:45
  • So a core dump only dumps the applications virtual memory, and does not affect other running processes – Omar Elrefaei Apr 25 '18 at 9:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.