17

If I do (in a Bourne-like shell):

exec 3> file 4>&3 5> file 6>> file

File descriptors 3 and 4, since 4 was dup()ed from 3, share the same open file description (same properties, same offset within the file...). While file descriptors 5 and 6 of that process are on a different open file description (for instance, they each have their own pointer in the file).

Now, in lsof output, all we see is:

zsh     21519 stephane    3w   REG  254,2        0 10505865 /home/stephane/file
zsh     21519 stephane    4w   REG  254,2        0 10505865 /home/stephane/file
zsh     21519 stephane    5w   REG  254,2        0 10505865 /home/stephane/file
zsh     21519 stephane    6w   REG  254,2        0 10505865 /home/stephane/file

It's a bit better with lsof +fg:

zsh     21519 stephane    3w   REG          W,LG  254,2        0 10505865 /home/stephane/file
zsh     21519 stephane    4w   REG          W,LG  254,2        0 10505865 /home/stephane/file
zsh     21519 stephane    5w   REG          W,LG  254,2        0 10505865 /home/stephane/file
zsh     21519 stephane    6w   REG       W,AP,LG  254,2        0 10505865 /home/stephane/file

(here on Linux 3.16) in that we see fd 6 has different flags, so it has to be a different open file description from the one on fd 3, 4 or 5, but from that we can't tell fd 5 is on a different open file description. With -o, we could also see the offset, but again same offset doesn't guarantee it's the same open file description.

Is there any non-intrusive1 way to find that out? Externally, or for a process' own file descriptors?


1. One heuristic approach could be to change the flags of one fd with fcntl() and see what other file descriptors have their flags updated as a result, but that's obviously not ideal nor fool proof

  • This approach should work, in principle, and not be too disruptive in most scenarios: first fork a child (with ptrace if doing it from the outside). Then, in the child, do something with the file descriptor that doesn't affect other processes. On Linux, leases should work for that. – Gilles Mar 23 '15 at 12:25
  • @Gilles, thanks but that's more or less the approach I suggest in the question already. leases (assuming you mean the F_SETLEASE fcntl, thanks for making me aware of them BTW) will only work for regular files you own and not if there's another "write" open file description to the same file (EBUSY), and it's not exactly non-intrusive. – Stéphane Chazelas Mar 23 '15 at 12:53
  • Have you abandoned this question? I posted some info regarding how SystemTap could do what you want, but you haven't marked any answer as complete...? – Azhrei May 14 '15 at 5:34
2

For Linux 3.5 and onward, this can be accomplished with kcmp(3):

KCMP_FILE

  • Check whether a file descriptor idx1 in the process pid1 refers to the same open file description (see open(2)) as file descriptor idx2 in the process pid2. The existence of two file descriptors that refer to the same open file description can occur as a result of dup(2) (and similar) fork(2), or passing file descriptors via a domain socket (see unix(7)).

The man page provides an example specifically for the use case OP asked. Note that this syscall requires the kernel be compiled with CONFIG_CHECKPOINT_RESTORE set.

  • Thanks. Exactly what I was look for. Note that unless you're superuser, it has to be two processes of yours (and not be setuid/setgid...) (understandably) – Stéphane Chazelas Oct 2 '18 at 6:31
  • @StéphaneChazelas Exactly. If for some reason the CPIU support wasn't built in your kernel and you don't want to rebuild it, then I suppose you can always write a kernel module that exports some userland interface that lets you compare struct file * pointers. – minmaxavg Oct 2 '18 at 13:52
3

What you're looking to compare are the struct file pointers that the file descriptors point to. (Inside the kernel is one task_struct data structure for each thread. It contains a pointer to another structure called the files_struct. And that structure contains an array of pointers, each one to a struct file. It's the struct file that holds the seek offset, the open flags, and a few other fields.)

I don't know of any user-visible way to see the pointers in the files_struct other than the use of some intrusive tools. For example, SystemTap could be given a PID and it could find the corresponding task_struct and follow the pointers. If you're looking for passive, though, I think that's about it. Dell released a tool a long time ago called KME (Kernel Memory Editor) that gave a spreadsheet-like interface to live kernel memory and it could do what you want, but it was never ported to 64-bit. (I tried and never got it completely working, and wasn't sure why.)

One reason you're not finding lsof to be helpful is that it doesn't see those pointers either (but look at the +f option for non-Linux systems). You could theoretically compare all of the fields in the struct file and think the two structures are the same, yet they could still be from separate open(2) calls.

Take a look at the pfiles SystemTap script for ideas. If you modified it to print the address of the struct file, you'd have your solution. You might also check opened_file_by_pid.stp since there's a function in it that walks the files_struct, ie. the file descriptor table, looking at the struct file objects...

Might I ask what you're trying to accomplish?

  • I have to admit I can't remember the very case where I needed that. Some debugging or forensic task no doubt. – Stéphane Chazelas May 14 '15 at 8:31
  • I'm looking forward to the PoC systemtap code :-) – Stéphane Chazelas May 14 '15 at 8:31
  • Before I posted the question, I did have a look at systemtap or /proc/kcore approaches. The difficult part was to get the info for every fd of every task. The most promissing approach I did find was hooking into the functions that generate the content of the /proc/*/task/fd directory, but the only workable things I could come up with involved hooking at specific line numbers in the source file so not portable from one kernel version to the next. You can't really loop through the task list in systemtap. Maybe possible via /proc/kcore, but too much effort and probably unreliable. – Stéphane Chazelas May 14 '15 at 8:34
  • Thanks for the best response so far. I'll have a look at your pointers. – Stéphane Chazelas May 14 '15 at 8:35
  • Sure you can! Set up a probe begin block and have it use the for_each_process macro in a block of C code embedded in the script (you'll need to use "guru" mode SystemTap to embed C code). In fact, to make this interesting (!), you could use one of SystemTap's associative arrays; use the files_struct address as key, and a list of PIDs/TIDs as values. You now have a list of every opened file and which tasks are sharing them (they can be shared between parent/child). Reply again if you want to discuss SystemTap. – Azhrei May 18 '15 at 3:26
0

Here is a linux specific solution: /proc/self/fd is a directory of symbolic links for open file handles in the current process. You can just compare the link values. It gets more complicated when using a child process, because the child will have a different /proc/self because it is a pid dependant symbolic link. You can workaround this problem by using /proc/$$/fd where $$ is the desired pid.

  • Thanks. But that's not what I'm asking. On Linux, lsof does indeed use /proc/pid/fd to retrieve paths for each file descriptor and /proc/pid/fdinfo for the flags. But what I want is that for two fds to the same file whether they point to the same open file description or if the two file descriptors have been open independantly. – Stéphane Chazelas Apr 25 '15 at 19:40
  • ok, after you have found pairs of file descriptors that are open to the same file name, do a tell on both and compare results, if they differ they are separate. If they are the same seek on one file descriptor and repeat, If they still match they are the same. – hildred Apr 25 '15 at 19:55
  • Well, that's a more intrusive variant of the heuristic approach I refer to in the question and that only works for regular files (not sockets, devices (like terminals), pipes...). – Stéphane Chazelas Apr 25 '15 at 20:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.