A reference to the esoteric details needed for building dynamically linked software at the native layer

Operating Systems: Windows, Linux

Compilers: CL (Microsoft C/C++ Optimizing Compiler), GCC, CLANG

Arcane Arts

If you are already familiar with the principle ideas of dynamic linking, then my dynamic linking example repository can help fill in all the details for putting dynamic linking into practice.

It contains an assortment of examples that cover the depths of the topic. It has concrete examples of dynamic linking on Windows and Linux, with CL, GCC, and CLANG, additional concrete examples for solving the "initialization problem", and an example abstraction that creates a portable, dynamically linkable, base layer.

At the root of the repository you will find a GUIDE.txt that will help you get your bearings in the repository. While you are learning, take the time to read the code and the DETAILS.txt in each example. The code and build scripts are designed to be as clear and minimal as possible, so you can run and modify them to see them working. On the other hand, the DETAILS.txt are written to cover the examples ideas thoroughly.

Once you've understood these concepts and tools, the rest of the reference is organized so that you can use it to quickly refresh your memory with the keywords, command invokations, and terminology.

Incantations

Incantations with CLANG

Mark a function as an exported symbol

Mark a function as a load-time imported symbol

Mark a function as a local symbol

GCC symbol visibility defaults

Generating shareable binaries (.dll/.so)

Generating binaries with load-time imported symbols

cl [my_program].c [load_time_dependency].lib
link [my_object1].obj [load_time_dependency].lib

gcc [my_program].c [load_time_dependency].so
ld [my_object1].o [load_time_dependency].so

Custom loader paths

Includes for run-time loading & linking

API for run-time loading

API for run-time linking

Execution on load before main

#pragma section(".CRT$XCU", read) per compilation unit
__declspec(allocate(".CRT$XCU"))
__pragma(comment(linker,"/include:unique-name"))
void(*unique-name)(void) = your-function;

__attribute__((constructor)) your-function-definition

Lexicon

The terminology in this topic is not exactly standardized. I think the way I use terms makes sense and lines up pretty well with common usages I've seen. I use these terms throughout the reference so I include them here for clarity.

I use dynamic linking to mean any architecture in which a process works by loading more than one binary. Put concretely, linking to 'dynamic link libraries' and 'shared objects' (.dll files and .so files).

I use binary file to mean a file that contains executable data. These generally come in two 'flavors'. An executable is a binary file which a user can execute directly, and starts a new process. The other 'flavor' is a little harder to name. On Windows it is the .dll or dynamic link library. On Linux it is the .so or shared object. When I want to be generic I'll just call it a shareable binary.

I use process to mean the operating system construct that organizes the ongoing activity of a single program. A process can be pieced together out of multiple binary files. Usually one executable and zero or more shareable binaries. However, we could load multiple executables if we wanted to. The only real rule is that to launch a process the initial binary file has to be an executable and this limitation is just a feature of the command line interface and the file explorer interface.

I use linker to mean a program which generates binary files.

I use loader to mean the part of the operating system which reads data out of binary files and places that data into a process. On Windows this part of the operating system is totally opaque to us. On Linux, for better or worse, this part of the operating system is extremely flexible, and can be redefined in a number of ways.

I use linking to mean correlating usage sites with definition sites. In this context, usage sites are usually function calls, and definition sites are function bodies. We can also generate a usage site in this context by using a function name as a function pointer. This is not only performed in the linker. Linking happens at many stages of the translation from source code to a binary file, it also happens in the loader, and even in our own code as we build our software.

I use link-time linking to mean the linking that occurs inside of a linker. Linking that occurs here is baked into the binary file that the linker produces.

I use load-time linking to mean the linking that occurs inside of a loader when a process is first being initialized. This occurs when the initial executable references other binaries as load-time dependencies. Each binary file can include a list of it's own unresolved symbols, that need to be resolved by load-time linking. If everything works as it's supposed to, the list of load-time dependencies in each binary file will ensure that every all the symbols in this unresolved list get resolved. If there are any load-time dependencies the operating system can't locate, it produces an error message and kills the process.

I use run-time loading to mean the loading of binary files inside our own code. This goes through APIs like LoadLibraryA and dlopen. When a binary file is loaded at run-time the operating system still tries to resolve its load-time dependencies. If the operating fails to find the binary file or it fails to load for any reason, the process continues, our process gets the opporunity to handle the failure to load the binary in our own way.

I use run-time linking to mean the linking that occurs inside our own code, after run-time loading is successful. This goes through APIs like GetProcAddress and dlsym.

I use load-time imported symbol to mean a symbol that is marked as unresolved inside a binary file. These exist in a list in the binary file. All load-time imported symbols must be linked by the loader itself, or the load process fails.

I use exported symbol to mean a symbol that is defined within a binary file and marked as visible to the outside world by the binary file. These exist in a list in the binary file. An exported symbol can be used by the loader during load-time linking or it can be used by our own code during run-time linking.

I use local symbol to mean a symbol that is defined within a binary file and that is not visible to the outside world. These symbols may be available in link-time linking but they are not preserved in any data structure in the final binary file. The artifacts of these symbols which remain are the machine instructions and other data generated by the compiler and linker, but no trace of the symbol name remains, so they cannot be located and used by later linking processes.