I have left a script running on a remote machine from when I was locally working at it. I can connect over SSH to the machine as the same user and see the script running in ps.

$ ps aux | grep ipcheck
myuser  18386  0.0  0.0  18460  3476 pts/0    S+   Dec14   1:11 /bin/bash ./ipchecker.sh

It is simply outputting to stdout on a local session (I ran ./ipchecker.sh form a local terminal window, no redirection, no use of screen etc).

Is there anyway from an SSH session I can view the output of this running command (without stopping it)?

So far the best I have found is to use strace -p 18386 but I get hordes of text flying up the screen, its far too detailed. I can stop strace and then sift through the output and find the text bring printed to stdout but its very long and confusing, and obviously whilst it's stopped I might miss something. I would like to find a way to see the script output live as if I was working locally.

Can anyone improve on this? The obvious answer is to restart the script with redirection or in a screen session etc, this isn't a mission critical script so I could do that. Rather though, I see this as a fun learning exercise.

  • Is your process running in a virtual console or in a GUI/xterm like environment? – jippie Dec 15 '12 at 15:20
  • 1
    You can limit output of strace to one syscall: strace -p 4232 -e write – otokan Dec 15 '12 at 15:59
  • @jippie The machine is running a full GUI (Linux Mynt 13, XFCE desktop), I fired up a gnome-terminal. – jwbensley Dec 15 '12 at 16:14
  • 3
    There's at least a dozen of similar questions on this site. Look for reptyr here to find a few of them (and an answer). – Stéphane Chazelas Dec 15 '12 at 22:46

If all you want to do is spy on the existing process, you can use strace -p1234 -s9999 -e write where 1234 is the process ID. (-s9999 avoids having strings truncated to 32 characters, and write the system call that produces output.) If you want to view only data written on a particular file descriptor, you can use something like strace -p1234 -e trace= -e write=3 to see only data written to file descriptor 3 (-e trace= prevents the system calls from being loged). That won't give you output that's already been produced.

If the output is scrolling by too fast, you can pipe it into a pager such as less, or send it to a file with strace -o trace.log ….

With many programs, you can divert subsequent output with a ptrace hack, either to your current terminal or to a new screen session. See How can I disown a running process and associate it to a new screen shell? and other linked threads.

  • 14
    I don't suppose there's a way to narrow down the output to just the standard output? – Jonah Mar 28 '13 at 0:33
  • 3
    Can you explain all the arguments? – User Mar 16 '15 at 22:15
  • 5
    There was a lot of backslashes and numbers in a lot of the output I was getting from nodejs; Any leads on what encoding they might be in? There was plenty of plain text too, which was all i needed. – ThorSummoner Jul 16 '15 at 17:27
  • 3
    "Can you explain all the arguments?" @User: man strace – Pistos Sep 1 '15 at 15:32
  • 3
    @RafaelMoni A program to do what you're asking is called a debugger. – Gilles Dec 22 '15 at 12:12

You can access the output via the proc filesystem.

tail -f /proc/<pid>/fd/1

1 = stdout, 2 = stderr

  • 4
    It gives me: 'cannot open /proc/<my pid>/fd/1 for reading: No such device or address'. – Yaroslav Nikitenko Oct 27 '16 at 13:36
  • 12
    yeah <my pid> should be your process id – tvlooy Nov 8 '16 at 14:10
  • 13
    This won't work if the output is going to a tty (or redirected to /dev/null) — it will only work if the output is redirected to a file. – mattdm Apr 26 '17 at 18:19
  • The output of yum is not loading, coz it is downloading some file from a repo and the progress is not showing under the file 1 – Ranvir Oct 5 '17 at 4:41
  • 1
    Tested on Ubuntu 16.04 and it doesn't work. In a session I do: ping google.es and in another one as root: tail -f /proc/`pgrep ping`/fd/2 and nothing is shown. – david.perez Mar 2 at 8:57

In BSD, you can use watch which snoops a given tty, e.g.

watch /dev/pts/0

In Linux, it won't be possible if the process wasn't run under multiplexer before such as screen or tmux. See also: Reptyr: Attach a Running Process to a New Terminal

It seems the only way is to debug the process (e.g. strace, dtrace/dtruss, gdb, lldb, etc.).

Since you've used strace, to fetch any meaningful output, you need to filter by a qualifying expression (such as file), then parse the output. Here is example:

strace -e trace=write -s1000 -fp 18386 2>&1 | grep -o '".\+[^"]"'

What it does it prints write operation of the process (1000 length) specified by PID (use pgrep to find it by name), redirects standard error into output (to be filtered), and prints double-quoted string.

If you're dealing with binary output, you may parse escape sequences characters by using read (with -r) and printf (with %b), e.g.

while read -r -t1 line; do printf "%b" $line; done

Check help read for more parameters (e.g. -n to print after certain amount of characters, rather than newline).

Here is more complete example:

strace -e trace=write -s1000 -fp 18386 2>&1 \
| grep --line-buffered -o '".\+[^"]"' \
| grep --line-buffered -o '[^"]\+[^"]' \
| while read -r line; do
  printf "%b" $line;
done

For examples using any process, please check: How to parse strace in shell into plain text? at stackoverflow

  1. You might be able to peek at the remote screen using ssh localhost 'DISPLAY=:0.0 xwd -root' | xwud -scale where localhost is to be replaced by your remote server login credentials and :0.0 with the display number of your GUI.

  2. Use x11vnc, which is a VNC server for your on screen X-session.

  3. When running on one of the 6 virtual consoles try sudo setterm -dump 2 -file /dev/stdout, where you replace 2 with the appropriate vc.

I'd advise to make a named pipe (mkfifo) and then write to that file. Then, read from it. You can always do that with things like tail, to minimize output, etc. Whenever you clear the pipe (read from it), it gets cleared, so the output is not preserved.

The other option would be to write everything to a file (much like a logfile) and then analyze it an any time. This would be the preferred action, if you want to preserve all output.

You always can launch a process whith nohup and &

nohup rsync source_file dest_file &

Then, you'll can check the progress from any tty with:

tail -f nohup.out

This works fine to me.

  • 5
    This question I'd about how to view the output of an alert running process, not how to run a process in the background as your answer suggests – jwbensley Apr 26 '17 at 18:18

A very simple to get the output would be capturing your output to a file and tailing that file.

IF DO: ./ipcheck

INSTEAD DO: ./ipcheck > [replacewithyourfilename]

This would create an output file where your script is located. Then from any other bash shell you can simply tail the file:

tail [replacewithyourfilename] -f

  • File redirection tends to use block-based buffering by default (see setbuf(3)) which might make a tail problematical. – thrig Apr 26 '17 at 18:14
  • 4
    Also this doesn't help with the question, which was about how to view the output of an already running process, not how to redirect stdout for a new process. – jwbensley Apr 26 '17 at 18:19

can't you get the process ID and communicate with it with USR1,

$pgrep -l '^ipchecker.sh$'

which prints your script's PID, then use it to

$ kill -USR1 PID

I understand that USR1 is a "user defined" signal, meaning that whoever created the program can use it to mean "shut down" or "dump your logs" or "print foo a thousand times" or whatever.

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.