2

I have a few scientific programs that can run from a few minutes up to several days. The program prints progress information to stdout which ends in a carriage return.

If the program runs for several days I usually pipe the output to /dev/null to avoid huge logfiles.

Is there a way to periodically watch the output of the program, without having to keep all of the generated text?

4
  • What's wrong in running it in a terminal and watch anytime? Commented Apr 12, 2022 at 8:37
  • The program is often run via ssh on a compute server. I'd like to be able to check in on it, even when it's started in the background
    – Kilian
    Commented Apr 12, 2022 at 9:10
  • 2
    Can you use tmux or screen on the server? Commented Apr 12, 2022 at 9:15
  • I can! Thanks, that should enable me to do what I want. @KamilMaciorowski
    – Kilian
    Commented Apr 12, 2022 at 9:37

2 Answers 2

4

You should ssh to the server, then run screen to start a session. From there, launch your long-running scientific program (leaving it output normally to stdout), then disconnect anytime. Whenever you feel like watching the output, ssh back to the server again and run screen -R to reattach to the last detached session.

You can also use tmux instead of screen, it works similarly. Just run tmux attach-session instead of screen -R.

To do the same with an instance of your scientific program that was not already started in a screen session: find the program PID, then run:

screen
reptyr <PID>

to reattach it to your new screen session.

1
  • 1
    Note: reptyr may require adjusting /proc/sys/kernel/yama/ptrace_scope. If reptyr fails for you then see this answer. Commented Apr 12, 2022 at 18:18
0

I just developed a new solution in this exact problem area: the pw (Pipe Watch) utility.

This is a small, BSD-licensed C program, less than 2000 lines in a single file, with no library dependencies, producing a small executable with a small memory footprint.

pw constantly reads its standard input, and so it eats everything, just like /dev/null. It passes the lines of input through an internal FIFO buffer, which is sampled to create an updating display.

When pw detects that it has been backgrounded by shell job control, it still reads its input, but doesn't refresh the display.

It's really as easy as:

yourlongrunningprogram | pw &

Then fg it into the foreground to view the latest output; Ctrl-Z and bg to run in the background again.

pw has some nice interactive features: filtering, triggering, split screen operation, recall of snapshot history. The user interface is memetic of Vi: there are : colon commands and such. Some commands have numeric prefixes, and repeating a command is done with ..

Say you've been running your program for several hours or days and decide that some parts of its output are not informative. No problem, just :v pattern and lines not matching pattern are now rejected.

The most recently developed feature is that you can save the state of pw to a file. If you interactively set it up in a certain way, you can easily reproduce that setup from the saved file: all your filters, triggers, split screen config and other parameters.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .