Ali Bahrami: Solaris Linkers and Linking Accessories
| Ali Bahrami: Solaris Linkers and Linking Accessories | |
The new kldd ELF utility brings ldd style analysis to kernel modules.
Solaris 11.4 comes with a number of enhancements that allow the elfdump utility to display a wealth of information that was previously hidden in Solaris core files. Best of all, this comes without a significant increase in core file size.
Starting with Solaris 11.4, program headers in Solaris ELF objects have explicit names associated with them. These names are used by libproc, elfdump, elfedit, pmap, pmadvise, and mdb to eliminate some of the guesswork that goes into looking at process mappings.
The simple ELF rules that let programs like strip renumber sections and fix up the sh_link and sh_info section header fields, without understanding the content of the sections involved.
The ways in which more than one ELF object with the same SONAME can end up in a process, what the linkers do to try and prevent it, why that's not completely possible, and why this situation is something to avoid rather than try and manage.
In cooperation with the GNU community, we are happy and proud to bring standard ELF section compression APIs to libelf. This builds on our earlier work in 2012 (Solaris 11 Update 2) to standardize ELF compression at the file format level. Now, others can easily access that functionality.
Solaris Kernel Modules (kmods) are now explicitly tagged as such, and are treated as final objects.
Pattern matching using regular expressions, globbing, or plain string comparisons, bring new expressive power to Solaris mapfiles.
Publically documented and committed CRT objects for Solaris.
A long awaited simplification to the process of building multithreaded code, one of the final projects delivered to Solaris by Roger Faulkner, made possible by his earlier work on thread unification that landed in Solaris 10.
Weak Filters allow the link-editor to discard unnecessary libc filters as dependencies, because you can't always fix the Makefile.
In Solaris 11 Update 4 (and Solaris 11 Update 3), the 32-bit version of the link-editor, and related linking utilities, are gone.
The Solaris 11.4 Public Beta has been released!
A description of Solaris Parent Objects, which were introduced in Solaris 11 Update 1. Parent objects allow plugin objects to be built, with the ld -z defs option, linking against the parent, so that external symbols from the parent can be seen at link time, producing a completely linked object that won't fail to find the external symbols it needs at runtime. This is often referred to as "symbol closure". A side bonus is that the file utility can identify the parent for plugin objects.
Ancillary Objects, an ELF feature introduced with Solaris 11 Update 1, allow the debug sections for an object to be written to a separate file that need not be present for the object to be used at runtime.
The standard file utility is inconvenient when applied to archives of ELF objects. Introducing elffile, a Solaris utility that provides the same sort of information as file, without requiring the contents of the archive to be unpacked.
Once we had the stub proto in place for building Solaris, we found additional unanticipated uses for it.
How to use stub objects to simplify builds, increase build parallelism without complicated makefiles or ordering, and improve object interfaces. Stub objects changed the game for us in terms of build reliability, and the overall build rules are simpler too.
Solaris 11 delivered a new type of object, known as a stub. A stub object is an object, built entirely from mapfiles, that supplies the same linking interface as the real object, while containing no code or data. This post describes the long process from idea to practical reality used every day to build the Solaris OS.
Like most long lived tools, the linkers have some unfortunate defaults. In addition, there are newer abilities that people don't always use. How do you push the world to use that stuff, without breaking existing applications?
How Solaris ELF archives were extended so that ELF archives can exceed the 32-bit 2GB file size limit, with a discussion of how other Unix like systems have tackled the same problem.
Solaris 11: A few new ELF and linking features: 64-bit archives, ld guidance, stub objects, elffile.
The rules that govern how Solaris shared objects are named, with a deep dive on how SONAMEs and compilation symlinks conspire to let you link with the latest version of a library, while old programs continue to run with the old versions of the same library.
An overview of the new version 2 Solaris Mapfile Language. Version 2 mapfiles are human readable, and provide a basis for the addition of future link-editor features in ways that the original language can't support.
I was a Sun user and ISV for many years before joining the Solaris group, and I wrote more than my share of low level C code, and used all the various compilers, linkers, and debuggers. And yet, I never fully understood the cryptic and odd Solaris link-editor mapfile language that was inherited with Sysv Unix from AT&T. When I joined Sun, I got the job of designing its replacement, and had to sit down and fully map it out. I realized that if I didn't write it down, I'd forget it all again, as it's just that cryptic. The best way to use this is to decode your existing mapfiles and then rewrite them in the the version 2 mapfile language.
Solaris doesn't use the GNU hash, but I wanted to understand it in enough detail to be able to implement it, and while I found bits and pieces of what I needed on the web, there was no single detailed description. I ended up contacting the authors to clarify and explain some bits. They were generous with their time, and helped me out. In return, here is a detailed description of the GNU hash, with enough background to understand it.
Standard ELF uses hash sections that are essentially unchanged since they were defined at AT&T in the 1980's. GNU have created their own replacements for these, in the interest of speeding up the hash process. Here, I consider the costs involved in ELF hashing, and highlight the difference in approach between Solaris, which uses direct bindings to drive down these costs (as well as other benefits), and the GNU hash which seeks to drive out all the unnecessary computational overhead of classic ELF symbol lookup.
The Solaris link-editor (ld) has the ability to link objects for a target machine different than the one doing the link. As an example, you can link sparc objects while running on an x86 based system.
The LD_LIBRARY_PATH environment is a necessary evil that is used far more often than it should be. With the addition of the elfedit utility, I revisit this old topic, describing the many ways in which LD_LIBRARY_PATH can bite, provide some strategies for reducing its use, and some guidelines on how to manage its use if you must use it.
elfedit is a new Solaris utility that allows for arbitrary in place editing of ELF objects. It's a boon for debugging, and testing, and early experimentation, and it finally provides a way for people who are not comfortable editing ELF objects in emacs to alter the runpath of an object. It's an expert tool, so it can also break objects, which is in itself useful in a development environment.
elfdump gains the ability to ignore the section headers in and ELF object, and instead synthesize a set based on the program headers. This can be useful if the object has been damaged, or if you suspect that it's been tampered with maliciously, because unlike section headers, program headers reflect the information actually seen by the runtime linker, and therefore reveal "truth" that might be obscured in tampered section headers. It's an obscure and hopefully rarely used feature that's nice to have in the toolbox.
One of the goals that led to the development of the elfedit utility was to enable modifying the runpath of an existing object without having to relink it. The changes to Solaris ELF objects required to support that are described here, and a simple program to do that named rpath is provided. Today we have elfedit, so there's no reason to use rpath, but it still serves as documentation.
In the previous posting about the new .SUNW_ldynsym sections, I claimed that Solaris objects have been shipped in unstripped form for many years. I learned later that this is only partially true, and that things were greatly improved with Solaris 10. This article gives the details.
Solaris ELF files have a new ELF symbol table. The section type is SHT_SUNW_LDYNSYM, and the section is named .SUNW_ldynsym. In the 20+ years in which the ELF standard has been in use, we have only needed two symbol tables (.symtab, and .dynsym) to support linking, so the addition of a third symbol table is a notable event for ELF cognoscenti. Even if you aren't one of those, you may encounter these sections, and wonder what they are for.
ELF symbol tables, and the rules and conventions that govern their layout.
An exploration of the mess that is Unix "tentative" symbols, also known as "common" symbols, and a tip on how to prevent them from happening in your code.
I used SunOS and later Solaris for many years before joining Sun. Joining the Solaris group allowed me to fix a classic misfeature of the link-editor, and cause output files to be unlinked before being created, rather than writing the new content in place.
In the past, a programming mistake in the runtime linker could immediately brick your system (render it unusable), and recovery was slow and tedious (boot the CD, put back the old runtime linker, reboot). Solaris Zones provide the option to employ a dirty and unsupported, but very useful trick for testing new development linker components without the risk of bricking the box.