6

I am running a Perl script on a Linux machine via a cron job. However, from time to time (around 1% of all cases), the script gets stuck and runs infinitely. If I list processes, I can see its PID. However, I don't want to kill it right away; I would rather know what went wrong.

Is there a way how to show what lines are being executed from the script? Something like step-by-step debugging of the script based on PID.

7

Try to follow these steps: - find the process pid of the shell, you may use a command like:

  ps -ef | grep <your_script_name> 
  • Let's set this pid in the shell variable $PID. Find all the child processes of this $PID by run the command:

    ps --ppid $PID
    

You might find one or more (if for example it's stuck in a pipelined series of commands). Repeat this command couple of times. If it doesn't change this means the script is stuck in certain command. In this case, you may attach trace command to the running child process:

    sudo strace -p $PID

This will show you what is being executed, either indefinite loop (like reading from a pipe), or waiting on some event that never happens.

In case you find ps --ppid $PID changes, this indicates that your script is advancing but it's stuck somewhere, e.g. local loop in the script. From the changing commands, it can give you a hint where in the script it's looping.

Finally, a very simple method to debug a perl is to use perl debugger:

perl -d script.pl

More: 1, 2, 3, 4, 5

  • Why are you looking for child processes of the script? perl doesn't need to run child processes unless you use things like system() to execute external commands. – Barmar Sep 22 '18 at 18:20
  • Did you misread the question to be asking about a bash script rather than perl? Your Stack Overflow links suggest so. – Barmar Sep 22 '18 at 18:22
  • Hi @Barmer. Thanks ;-) Indeed, I assumed that the perl relies of something like system() because it is very rare for simple programs to hang out like the OP description. For the link I am not sure what do you mean! – user88036 Sep 22 '18 at 18:48
  • 1
    The "More" links at the end are all about debugging bash, not perl. – Barmar Sep 22 '18 at 19:39
  • Tracing the perl process itself would show you which processes it's exec'ing, and then you'll see it sit in wait() while waiting for it to finish. But you should add the strace option to traces forked children. – Barmar Sep 22 '18 at 19:40
3

For the next runs of your script you can try out the package Devel::Trace.

From description: "This module will print a message to standard error just before each line is executed."

Run either with

perl -d:Trace program

or use in your script with

import Devel::Trace 'trace';

trace 'on';                 # Enable
trace 'off';                # Disable
  • But the output from the script will be captured by cron, so to actually see this you either need to look at the file descriptors used for that communication or convince cron that the script is done so it will send a mail with the output, hopefully killing it will be enough for that, but I've never tried. And as the OP said it was only about 1% of the invocations that didn't terminate in time, this could generate a lot of output before anything useful. I would probably try strace as in Goro's answer first. – Henrik - stop hurting Monica Sep 22 '18 at 9:59
  • @Henrik While debugging, you can run bash -c 'perl -d:Trace program 2>&1 | tee /tmp/program.$$' to capture the output. And it doesn't matter much if many of generated files here are useless. The strace approach is universal but the output there must be harder to match with the actual Perl code. – Kirill Bulygin Sep 22 '18 at 12:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.