There are 2 low level tools you can use to do this,
gdb. The availability of
strace is assuming you're on linux or OS where it works. For other OSs, you might have
dtrace, but the technique is similar.
strace is easier to use than
gdb, and usually answers the question of "what is this application doing" just fine. Typical usage (for me) would be something like:
strace -f -tt -s 200 -p $PID
The only real option you need is the
-p $PID. The other options just add more info to the output line, but aren't really necessary.
What this does is show you every system call the application is making. The only way an application can be doing something, and
strace not to show it, is if it's purely computational. Meaning crunching numbers or something. Things like reading/writing a file, sending a packet, getting the current time, etc, all require a system call and will show up.
gdb is lower level than
gdb will show you what line of code the application is actually running. However there are a few gotchas to this. The big one is that you need the debugging symbols for the application. On distro-provided packages, these have usually been stripped out. However on distros like RedHat derivatives, they are often provided in the form of
pkgfoo-debuginfo-1.2.3.rpm. You can just install this package and get the symbols back.
Another thing is that where
strace will show you what the application is doing over time,
gdb shows you what the application is doing at that exact moment, as it freezes the process while you are inspecting it.
So anyway, usage looks like:
gdb -p $PID
...which will drop you to an interactive shell where you can run something like:
I'm not going to go into the details of how to use
gdb, as depending on the complexity of your application, it could be complex, and there are plenty of guides on the internet.
When you're done poking at the process, and want to let it resume, just use
quit or CTRL+D.