2

I found this Q&A saying shared libraries can be shared between processes using shared memory. It seems like it would be impossible, though, to share code between processes without some pretty severe restrictions on the type of code that can be shared. I'm thinking about libraries with non-reentrant C functions whose output depends on the values of global variables or static variables inside their definition body. Like this one.

int really_really_nonreentrant(void x)
{
    static int i = 0;
    i++;
    return i;
}

A library with a function like this will return separate increasing sequences for each process using it, so it definitely seems like the code isn't being shared between processes. Is really_really_nonreentrant() being separated from the reentrant functions, or is it being kept mostly with the other functions with just static int i being separated out? Or is the entire library being kept out of shared memory because this function is nonreentrant?

Ultimately, how much does one have to do to ensure one's library gets allocated to shared memory?

  • Either you are confusing concepts, or the post needs some clarification. Often the virtual memory mechanism will share code in memory to save resources, the data segment/stacks will be distinct; the problem of reentrancy will present itself in the same program with multiple threads/points of entry. For instance, it is known printf is not reentrant, or the kernel has non-reentrant points/functions. Share memory per se is yet another concept; you might be thinking of shared libraries, which is yet another different thing. I might come back and turn this comment in a proper answer later on. – Rui F Ribeiro Mar 11 '16 at 12:15
  • I might not have the terminology exactly right, but I think the spirit of the question is pretty clear. Is non-reentrant code shared in memory between processes, and if so, how? – enigmaticPhysicist Mar 11 '16 at 12:28
  • I did not dismiss your thread, I simply havent got the time yet to give it a proper answer. – Rui F Ribeiro Mar 11 '16 at 13:29
3

I believe that the really short answer is that Linux compilers arrange code into pieces, at least one of which is just pure code, and can therefore be memory mapped into more than one process' address space. Any globals get mapped such that each process gets its own copy.

You can see this using readelf, or objdump, but readelf gives a clearer picture, I think.

Here's a piece of output from readelf -e /usr/lib/libc.so.6. That's the C library, probably mapped into almost every process. The relevant part of readelf output (although all of it is interesting) is the Program Headers:

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  PHDR           0x000034 0x00000034 0x00000034 0x00140 0x00140 R E 0x4
  INTERP         0x164668 0x00164668 0x00164668 0x00017 0x00017 R   0x4
      [Requesting program interpreter: /usr/lib/ld-linux.so.2]
  LOAD           0x000000 0x00000000 0x00000000 0x1adfc4 0x1adfc4 R E 0x1000
  LOAD           0x1ae220 0x001af220 0x001af220 0x02c94 0x057c4 RW  0x1000
  DYNAMIC        0x1afd90 0x001b0d90 0x001b0d90 0x000f8 0x000f8 RW  0x4
  NOTE           0x000174 0x00000174 0x00000174 0x00044 0x00044 R   0x4
  TLS            0x1ae220 0x001af220 0x001af220 0x00008 0x00048 R   0x4
  GNU_EH_FRAME   0x164680 0x00164680 0x00164680 0x06124 0x06124 R   0x4
  GNU_STACK      0x000000 0x00000000 0x00000000 0x00000 0x00000 RW  0x10
  GNU_RELRO      0x1ae220 0x001af220 0x001af220 0x01de0 0x01de0 R   0x1

The two LOAD lines are the only pieces of the file that get mapped directly into memory. The first LOAD header maps a piece of /usr/lib/libc.so.6 into memory with R and E permissions: read and execute. That's the code. Hardware features keep a program from writing to that piece of memory, so all programs can share the same pages of real, physical memory. The kernel can set up the hardware to map the same physical memory into all processes.

The second LOAD header is marked RW - read and write. This is the part with global variables that the C library uses. Each process gets its own copy in physical memory, mapped into that process' address space with the hardware permissions set to allow reading and writing. That section is not shared.

You can see these memory mappings in a running process using the /proc file system. A good command to illustrate: cat /proc/self/maps. That lists all the memory mappings that the cat process has, and from what files the kernel got them.

As far as how much you have to do to ensure that your function is allocated to memory that gets mapped into different processes, it's pretty much all down to the flags you give to the compiler. Code intended for ".so" shared libraries is compiled "position independent". Position independent code does things like refer to memory locations of variables with offsets relative to the current instruction, and jumps or branches to locations relative to the current instruction, rather than loading from or writing to absolute addresses, and jumping to absolute addresses. That means the "RE" LOAD piece of /usr/lib/libc.so and the "RW" piece only have to have be loaded at addresses that are the same distance apart in each process. In your example code, the static variable will always be at least a multiple of a page size apart from the code that references it, and it will always get loaded that distance apart in a process' address space due to the way the LOAD ELF headers are given.

Just a note about the term "shared memory": There's a user-level shared memory system, associate with "System V interprocess communications system". That's a way for more than one process to very explicitly share a piece of memory. It's fairly complicated and obscure to set up and get correct. The shared memory that we're talking about here is more-or-less completely invisible to any user process. Your example code won't know the difference if it's running as position independent code shared between multiple processes, or if it's the only copy ever.

  • That´s quite a nice post about executable files. I got lost in your explanation of libc.so in the paragraph before the last. Is not one of the advantages of dynamic libraries for one copy be shared among all processes using it at a time? – Rui F Ribeiro Mar 12 '16 at 8:48
  • @RuiFRibeiro - It is said that one of the advantages of dynamic libraries is code sharing. The way ELF shared objects work, the kernel can do that. The "RE" LOAD header parts can be shared all over. It should also be possible to map the global variables (the "R W" LOAD header) as copy-on-write, so that as long as some process has it mapped, the kernel can just set up a mapping, vs reading it in from disk. So even the globals can be shared in a way. – Bruce Ediger Mar 12 '16 at 14:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.