3

In all the cases, I have found any file which is open for reading or writing is displayed in lsof output. But today, I am trying to found a logfile that is opened for writing by a java process in lsof output. But I can't find it in the output. But the logfile is active and is being updated.

Does files opened by java or python etc. won't show in lsof output?

FYI,

# lsof /usr/local/qftest/log/qftest.log
# lsof -p 30732 -a +d /usr/local/qftest/log
# lsof -p 30732 -a +D /usr/local/qftest/log
# lsof | grep qftest.log
# ps -p 30732 -f
  qftest 30732 1 99 Sep09 ? 6-07:37:34 java -Xbootclasspath/p:/usr/local/qftest/qftest-3.5.7/lib/endorsed/xalan.jar:/usr/local/qftest/qftest-3.5.7/lib/endorsed/xerces.jar:/usr/local/qftest/qftest-3.5.7/lib/endorsed/xml-apis.jar:/usr/local/qftest/qftest-3.5.7/lib/endorsed/serializer.jar -Dqftest.home=/usr/local/qftest -Dqftest.versionhome=/usr/local/qftest/qftest-3.5.7 -Dqftest.display= -Xmx256m -Xms16m -classpath /usr/local/qftest/qftest-3.5.7/qflib/qftest.jar:/usr/local/qftest/qftest-3.5.7/qflib/qfshared.jar:/usr/local/qftest/qftest-3.5.7/qflib/qfdemo.jar:/usr/local/qftest/qftest-3.5.7/qflib/qflog.jar:/usr/local/qftest/qftest-3.5.7/qflib/qflib.jar:/usr/local/qftest/qftest-3.5.7/lib/endorsed/serializer.jar:/usr/local/qftest/qftest-3.5.7/lib/endorsed/xml-apis.jar:/usr/local/qftest/qftest-3.5.7/lib/endorsed/xerces.jar:/usr/local/qftest/qftest-3.5.7/lib/endorsed/xalan.jar:/usr/local/qftest/qftest-3.5.7/lib/truezip.jar:/usr/local/qftest/qftest-3.5.7/lib/poi.jar:/usr/local/qftest/qftest-3.5.7/lib/netty.jar:/usr/local/qftest/qftest-3.5.7/lib/looks.jar:/usr/local/qftest/qftest-3.5.7/lib/jxl.jar:/usr/local/qftest/qftest-3.5.7/lib/junit.jar:/usr/local/qftest/qftest-3.5.7/lib/jide-oss.jar:/usr/local/qftest/qftest-3.5.7/lib/jfreechart.jar:/usr/local/qftest/qftest-3.5.7/lib/jcommon.jar:/usr/local/qftest/qftest-3.5.7/lib/commons.jar:/usr/local/qftest/qftest-3.5.7/lib/pngencoder.jar:/usr/local/qftest/qftest-3.5.7/lib/png.jar:/usr/local/qftest/qftest-3.5.7/lib/jython.jar:/usr/local/qftest/qftest-3.5.7/lib/jniwrapper.jar:/usr/local/qftest/qftest-3.5.7/lib/jline.jar:/usr/local/qftest/qftest-3.5.7/lib/jansi.jar:/usr/local/qftest/qftest-3.5.7/lib/groovy-all.jar:/usr/local/qftest/qftest-3.5.7/lib/gnu.jar:/usr/local/qftest/qftest-3.5.7/lib/bsf.jar: de.qfs.apps.qftest.start.QFTestRunner -shellarg=-c -shell=/bin/sh -options=/usr/local/qftest/qftest-3.5.7/bin/qftest.options -logfile=/usr/local/qftest/log/qftest.log -licenseserver

EDIT 1: After reading this lsof doesn't return files open by the same user, I thought there could be a swap file created, but there are no REG files in the lsof -p 30372 output with write mode.

Also there are no other child processes created from this process.

But the output shows many files like the below opened for writing:

java    30732 qftest   40r     FIFO        0,6          490757310 pipe
java    30732 qftest   41w     FIFO        0,6          490757310 pipe
java    30732 qftest   42r     0000       0,11        0 490757311 eventpoll
java    30732 qftest   43r     FIFO        0,6          490757312 pipe
java    30732 qftest   44w     FIFO        0,6          490757312 pipe
java    30732 qftest   45r     0000       0,11        0 490757313 eventpoll
java    30732 qftest   46r     FIFO        0,6          490757314 pipe
java    30732 qftest   47w     FIFO        0,6          490757314 pipe
java    30732 qftest   48r     0000       0,11        0 490757315 eventpoll

I don't know what is eventpoll here, but I think it is reading the log from eventpoll and it seems the logfile is split into several (don't know what it is) and is writing. It's like multiple processes are reading events and writing to the same file in some way I really don't know.

Improve this question
2
  • 2
    I'm having trouble to reproduce. My suspicion falls on the fact that the file is opened for append, appended and closed, repeatedly. Could you run the full command through strace? e.g. strace java -Xbootclasspath ... -logfile=...| grep qfest.log. That should print a quick open()/write()/close(), if that suspicion turns to be true. It is pretty decent question btw.
    – grochmal
    Sep 11, 2016 at 17:50
  • You are right. It does close it immediately. [pid 8609] open("/var/logs/qftest.log", O_WRONLY|O_CREAT|O_APPEND, 0666) = 243 [pid 8609] fstat(243, {st_mode=S_IFREG|0644, st_size=6839936, ...}) = 0 [pid 8609] write(243, "7 (21:29:32.801) New I/O worker "..., 8192) = 8192 [pid 8609] write(243, "l,android=0,user=qft_UI-test103_"..., 370) = 370 [pid 8609] close(243)
    – GP92
    Mar 9, 2018 at 19:28

1 Answer 1

Reset to default
-4

Files opened by programs are visible in the kernel data structures used by lsof, data is buffered internally, and the filesystem is only forced to be up-to-date when the program closes the file.

PIPEs are used to pass data from one process to another, fundamental to UNIX/Linux/*BSD/...

eventpolls are used for synchronizatipn, monitoring multiple data sources (eg keyboard and network activity).

Start with 

man -k open
man -k pipe
man -k event
man -k poll
man -k select

And read those man pages.

You are not in a maze of twisty little passages, all alike. It only seems like it.

Please read the How To Ask A Good Question post.

Improve this answer
3
  • "data is buffered internally, and the filesystem is only forced to be up-to-date when the program closes the file." Do you mean that the file is opened in w mode every time the program needs to update and as the update could be of small size, it happens in fraction of secs that we may or may not see in the lsof ouput?
    – GP92
    Sep 11, 2016 at 16:49
  • Thanks for explaining about pipe and eventpolls, I thought these are only specific to this particular process. It seems these are seen in most of the processes.
    – GP92
    Sep 11, 2016 at 16:51
  • 1
    I did suspect poll() but that can only poll over file descriptors that have already been opened therefore the kernel data structure must be updated before polling. I could understand seeing a file in lsof that has been closed (i.e. poll() would keep the kernel list in place and lsof will pick it up) but not the other way around.
    – grochmal
    Sep 11, 2016 at 17:41

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .