Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a program (C++) that works with TCP sockets in multithreaded manner. The multithreading is intensive, about 100 threads (POSIX threads).

Sometimes, not quite sure when, the program gets terminated by SIGTERM. After some googling I found out that it is not normal for system to send SIGTERM. I've decided to look what happens if I ignore the signal. Now the system sends SIGKILL. I assume it tries with SIGTERM and when the application does not terminate, the system kills it.

I have tried running it in gdb, and got no signals.

I've run it in valgrind, no signals. No valgrind errors either. Memory consumption was normal, it seems I have no memory leaks. Upon exit it had 7Mb heap in use.

Nothing suspicious in /var/log/messages, /var/log/syslogd.

The system is Debian 2.6.32-5.

Basically, the question is why may the system send SIGTERM followed by SIGKILL to an arbitrary process? And how can I stop at that point to see what happens (gdb alters the behavior).

share|improve this question
    
Since you're needing C++ debugging help, this is really a programming question. I've flagged it for migration to SO. –  depquid Apr 3 '13 at 19:41
    
@depquid He's got a program receiving unexpected signals; that doesn't seem like a C++ problem –  Michael Mrozek Apr 3 '13 at 19:44
    
@MichaelMrozek You're right; when I first read it, I was focused on the debugging aspects (valgrind, gdb) and that made me think "coding". –  depquid Apr 3 '13 at 19:56

1 Answer 1

'system' is a very ambiguous term. If we're talking the kernel, the kernel will never send a SIGTERM. It will send SIGKILL when the OOM killer is invoked though.

The likely scenario is a script or something which has a bad pkill or killall command that is inappropriately matching your process. When you launch the command with gdb, it's process name & arguments are different, so it would look different to the pkill/killall.

share|improve this answer
    
Thank you for your answer! I have renamed the file to 1, and it got killed. I have also created a while(1) sleep(1); process with the name it had before, put it in the same folder and it was not killed. Both processes had no arguments. So I assume it's not the process name. –  Aneri Apr 4 '13 at 8:39
    
@Aneri interesting. At this point options are limited. If you have selinux available, I would try creating a selinux policy to log when any application sends a signal. This way you can figure out where it's coming from. Or add something to LD_PRELOAD (for the entire system) to override kill() and log when it's called. –  Patrick Apr 4 '13 at 13:39
    
thanks for the reply. I have created a file with a logging version of kill(), but it has shown no kill of my pid! It does log kills, I've tested it and saw udevd killing some processes and so on. But nothing killed my program. I had my lib in /etc/ld.so.preload, so it was active for the entire system. I do not know how to set the policy for selinux, but will try to find out. –  Aneri Apr 4 '13 at 19:33

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.