0

Operating System Concepts, by Silberschatz A., Galvin P. B., Gagne G. - Operating System Concepts, 9th Edition - 2012 says

8.1.4 Dynamic Loading

In our discussion so far, it has been necessary for the entire program and all data of a process to be in physical memory for the process to execute. The size of a process has thus been limited to the size of physical memory. To obtain better memory-space utilization, we can use dynamic loading. With dynamic loading, a routine is not loaded until it is called. All routines are kept on disk in a relocatable load format. The main program is loaded into memory and is executed. When a routine needs to call another routine, the calling routine first checks to see whether the other routine has been loaded. If it has not, the relocatable linking loader is called to load the desired routine into memory and to update the program’s address tables to reflect this change. Then control is passed to the newly loaded routine.

The advantage of dynamic loading is that a routine is loaded only when it is needed. This method is particularly useful when large amounts of code are needed to handle infrequently occurring cases, such as error routines. In this case, although the total program size may be large, the portion that is used (and hence loaded) may be much smaller.

Dynamic loading does not require special support from the operating system. It is the responsibility of the users to design their programs to take advantage of such a method. Operating systems may help the programmer, however, by providing library routines to implement dynamic loading.

8.1.5 Dynamic Linking and Shared Libraries

What kinds of files have "a relocatable load format" in Linux,

  • an ELF executable file,
  • a .so shared library file,
  • a kernel module,
  • a .o object files?

Can they all be dynamically loaded?

Does "a relocatable load format" in the quote mean .o object file, a kernel module, but not .so shared library file, according to:

The book doesn't mention anything yet about shared library until Section 8.1.5, so Section 8.1.4 seems to me that dynamic loading is not necessarily loading a shared library into a user program, but may load something else. Is that true?

The last paragraph of section 8.1.4 seems to say that programmers need to perform dynamic loading explicitly. Does it refer to dlopen()? What kinds of ELF files can dlopen() accepts as its first argument, a .so shared library, a .o object file, a kernel module, an executable ELF file?

Thanks.

  • Where is this strange text from? – schily Oct 17 '18 at 17:51
  • It looks like the author of that text does not really understand how dynamic loading works. – schily Oct 17 '18 at 17:55
  • @schily Thanks. I am wondering what a relocatable format means? Do an executable, .so shared library, .o object file all have a relocatable format? – Tim Oct 17 '18 at 18:12
  • 2
    It's unfortunate that someone has (presumably) violated the Copyright of the 2009 book by posting it online. "No part of this publication may be reproduced..." -- and apparently there are many other such files on that jufiles website. Is there any indication that the website has acquired "prior written consent" from the publishers? – Jeff Schaller Oct 17 '18 at 18:26
  • 1
    Perhaps, besides the written quote in question, a regular old reference to the book would do? Title, author, editon, date, ISBN? – Jeff Schaller Oct 17 '18 at 19:00
0

A shared library does not really need to be relocatable. AT&T used non-relocatable shared libraries in SYSvr3 around 1987.

The method used by AT&T was based on fixed addresses in the shared lib and the shared lib was installed in system memory (usable by any process) by a special system tool at startup. The programs that used such libraries have been linked against the fixed addresses that were used at install time. This method is very limited as there is a need for a global administrator who defines load addresses for various libs.

The method that is used today is based on research at Sun Microsystems in 1987 and was published in December 1987.

It is based on the mmap() facility in the kernel. Each program contains special startup code that maps the needed libraries that use relocatable code.

Each library contains a special table that contains offsets for various symbols in the library. Each program conains a table with function names and the binary does a Jump Subroutine to the table entry related to all functions needed.

That table entry contains a jump instruction to the runtime linker code and when it is used the first time, the runtime linker replaces that jump by a jump to the funtion in the shared library.

All this is handled by system software and not subject of the users program design.

BTW: the book claims that no program may become larger than the physical memory. This was true in the 1960s. IBM changed that in the early 1970s for their mainframes and BSD changed that in 1979 for UNIX with virtual memory management.

dlopen() may be used to load dynamic libraries "manually" at run time and thus allows to make that depend on what happened before. A sound playing program may e.g. decide to use dlopen to load a mp3 decoder after it discovers that a file argument refers to a sound file that uses mp3 encoding.

If you use dlopen, linking to functions in the loaded shared library does not happen automatically. You need to use dlsym() to get the address from the function name and then call the function by a pointer dereferencing.

You cannot dlopen a .o file as there is a need to have the _PROCEDURE_LINKAGE_TABLE_ and that is created by the special linking process that is needed to create a .so. file.

The method used by Microsoft is derived from the manual method that is based on dlopen. Microsoft however automatically creates code that works the way you do manually with dlopen. The related code is inside the static library with the same name as the so called dll. This static library is linked statically to the program and contains the trmapolin code that loads the library, retrives the function address and then calls the function pointer.

The Microsoft compiler always creates relocatable code, so even fully static libraries compiled with that compiler use relocatable code.

The claim that a routine is not loaded until referenced is wrong in the general case, it only applies to the dlopen case.

Libraries managed by the ELF runtime system may be listed via ldd, libraries managed manally by dlopen cannot be listed since their names may be a result of run time string processing.

  • Thanks. "the book claims that no program may become larger than the physical memory." It doesn't, see ch9 virtual memory. Let's try to understand the textbook. – Tim Oct 17 '18 at 18:46
  • Your quoted text contains: "The size of a process has thus been limited to the size of physical memory." – schily Oct 17 '18 at 18:49
  • That is in Ch8 which discusses memory management without virtual memory system. – Tim Oct 17 '18 at 18:51
  • OK, but this was not part of your quote. – schily Oct 17 '18 at 18:54
  • I think the quote in my post for dynamic loading still applies in virtual memory system. – Tim Oct 17 '18 at 18:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.