Finding Symbols - reducing dlsym() overhead

Rod Evans — Friday September 09, 2005

In a previous post, I'd explained how lazy loading provides a fall back mechanism. If a symbol search exhausts all presently loaded objects, any pending lazy loaded objects are processed to determine whether the required symbol can be found. This fall back is required as many dynamic objects exist that do not define all their dependencies. These objects have (probably unknowingly) become reliant on other dynamic objects making available the dependencies they need. Dynamic object developers should define what they need, and nothing else.

dlsym(3c) can also trigger a lazy load fall back. You can observe such an event by enabling the runtime linkers diagnostics. Here, we're looking for a symbol in libelf from an application that has a number of lazy dependencies.

    % LD_DEBUG=symbols,files,bindings  main
    .....
    19231: symbol=elf_errmsg;  dlsym() called from file=main  [ RTLD_DEFAULT ]
    19231: symbol=elf_errmsg;  lookup in file=main  [ ELF ]
    19231: symbol=elf_errmsg;  lookup in file=/lib/libc.so.1  [ ELF ]
    19231:
    19231: rescanning for lazy dependencies for symbol: elf_errmsg
    19231:
    19231: file=libnsl.so.1;  lazy loading from file=main: symbol=elf_errmsg
    ......
    19231: file=libsocket.so.1;  lazy loading from file=main: symbol=elf_errmsg
    ......
    19231: file=libelf.so.1;  lazy loading from file=main: symbol=elf_errmsg
    ......
    19231: symbol=elf_errmsg;  lookup in file=/lib/libelf.so.1  [ ELF ]
    19231: binding file=main to file=/lib/libelf.so.1: symbol `elf_errmsg'

Exhaustively loading lazy dependencies to resolve a symbol isn't always what you want. This is especially true if the symbol may not exist. In Solaris 10 we added RTLD_PROBE. This flag results in the same lookup semantics as RTLD_DEFAULT, but does not fall back to an exhaustive loading of pending lazy objects. This handle can be thought of as the light weight version of RTLD_DEFAULT.

Therefore, if we wanted to test for the existence of a symbol within the objects that were presently loaded within a process, we could use dlsym() to probe the process:

    % LD_DEBUG=symbols,files,bindings  main
    .....
    19251: symbol=doyouexist;  dlsym() called from file=main  [ RTLD_PROBE ]
    19251: symbol=doyouexist;  lookup in file=main  [ ELF ]
    19251: symbol=doyouexist;  lookup in file=/lib/libc.so.1  [ ELF ]
    ......
    19251: ld.so.1: main: fatal: doyouexist: can't find symbol

When dlsym() is used to locate symbols from a handle returned by dlopen(3c), all the dependencies associated with the handle are available to the symbol lookup. I always thought this was rather odd, and that dlsym() should only look at the initial object of a handle. In other words, if you:

    if ((handle = dlopen("foo.so", RTLD_LAZY)) != NULL) {
            fprt = dlsym(handle, "foo");

then intuitively the search for foo would be isolated to foo.so, and not include the multitude of dependencies also brought in by foo.so. But, that is not how our founding fathers developed dlsym(). I think we even considered changing the behavior once so that dlsym() would only search the initial object. But we soon found a number of applications fell over as they could no longer find the symbols they were used to finding.

In Solaris 9 8/03 we provided an extension to dlopen() with the new flag RTLD_FIRST. By using this flag the same series of objects are opened and associated with a handle. However, only the first object on the handle is made available for dlsym() searches.

Perhaps RTLD_PROBE and RTLD_FIRST can reduce your dlsym() overhead.

Published Elsewhere

[16] a source tour

Blog Index (rie)

[18] Init and Fini