Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a program that will perform a lengthy calculation which may take several days to be completed.

Is there a way to save the program's running state; so I will be able to turn off the computer and run the program from a saved state. Or if the program crashed for any reason I can restart it from a saved state (and not from the outset)?

share|improve this question
    
To prevent process hangup when you need to turn off your computer, you could hibernate instead. –  edvinas.me Apr 18 at 9:37

3 Answers 3

Making restartable snapshots of a process is very hard, because a process can have all kinds of interactions with the filesystem. As a rule, don't expect unix systems to support this. There have been unix variants with restartable core dumps, but I don't think this is the case on modern ones (they've become too complex).

Lisp systems typically have a dump command that creates a restartable image. So you could write your program in a Lisp dialect that supports dumping.

You can run your code in a virtual machine and use the virtual machine manager (e.g. VirtualBox) to create periodic snapshots. Depending on what your program does, this may or may not hurt performance.

The best solution is probably for you to build a snapshot feature into your program. For purely computational programs, this is often only moderatly difficult. In a multithreaded computational program, snapshot points are typically global synchronization points, where all the threads communicate. Try to structure your program as a bag of tasks, and make the entry point a dispatcher which starts a task whenever a processor is free. Upon receipt of a signal, the dispatcher waits for all current tasks to finish, saves the program states, and starts dispatching tasks again.

share|improve this answer

I'd suggest hibernating instead of shutting down your computer, but if you really cannot avoid this there exists a snapshot/restore system called CRIU (Checkpoint/Restore in Userspace), that could suit your needs. With this framework you could quite easily »freeze« your processes (actually saving their state to disk) and later defrost them again.

I didn't extensively test this yet, but they seem to have some rather good documentation, so you might find that helpful. Also note that for using CRIU you might have to enable several kernel features which most probably aren't enabled on most stock distributions (CONFIG_CHECKPOINT_RESTORE and CONFIG_MEM_SOFT_DIRTY), so chance is good you'll need to compile your own kernel to get it running.

Note: If your program crashes for a reason freezing it beforehand is probably not the way to go, since most probably your program will run into the same conditions again (in case it's no external reason like full disk or memory).

share|improve this answer

Or if the program crashed for any reason I can restart it from a saved state (and not from the outset)?

First of all, it is much better if you analyze a core file and fix a bug that leads to a crash. As for your question you can use gdb and its checkpoints. Documentation is here sourceware.org/gdb/onlinedocs/gdb/Checkpoint_002fRestart.html

Let me show you an example:

OK, this is a program that runs for a long time and crashes at some point:

>more main.cpp
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>

int
main()
{
  int iterations = 0;
  while (iterations < 10)
  {
    printf("pid: %d, Iteration: %d\n", getpid(), iterations);
    ++iterations;
    sleep(5);
    if (iterations == 7) {
      int *ptr = 0;
      *ptr = *ptr+1;
    }
  }

  printf("exit\n");
  return 0;
}

I run it under gdb, but you can attach to a running process of course. So once I run checkpoint in my script in order to be able to return to a previous state. You can do it many times.

>more script.gdb
handle SIGSEGV stop print nopass
b main.cpp:12 if iterations==4
commands
checkpoint
c
end

Ok, I run my program. As you can see after "Iteration: 6" it crashes! Despite that it restart it from a saved checkpoint. See, my variable iteration is again 5 :

>gdb -q -x script.gdb main
Reading symbols from /main...done.
Breakpoint 1 at 0x40069e: file main.cpp, line 12.
(gdb) r
Starting program: /main
pid: 13744, Iteration: 0
pid: 13744, Iteration: 1
pid: 13744, Iteration: 2
pid: 13744, Iteration: 3
pid: 13744, Iteration: 4

Breakpoint 1, main () at main.cpp:12
12          ++iterations;
pid: 13744, Iteration: 5
pid: 13744, Iteration: 6

Program received signal SIGSEGV, Segmentation fault.
0x00000000004006be in main () at main.cpp:16
16            *ptr = *ptr+1;
(gdb) info checkpoints
  1 process 13756 at 0x40069e, file main.cpp, line 12
* 0 process 13744 (main process) at 0x4006be, file main.cpp, line 16
(gdb) restart 1
Switching to process 13756
#0  main () at main.cpp:12
12          ++iterations;
(gdb) c
Continuing.
pid: 13756, Iteration: 5
pid: 13756, Iteration: 6

Program received signal SIGSEGV, Segmentation fault.
0x00000000004006be in main () at main.cpp:16
16            *ptr = *ptr+1;
(
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.