The Link-editors - a source tour

Rod Evans — Tuesday June 14, 2005

Welcome to OpenSolaris. I've been working with the link-editors for many years, and I thought that with the general availability of the source, now would be an opportune time to cover some history, and give a brief overview of the link-editors source hierarchy.

The link-editor components reside under the usr/src/cmd/sgs directory. This Software Generation Subsystem hierarchy originated from the AT&T and Sun collaboration that produced Solaris 2.0. Under this directory exist the link-editors, and various tools that manipulate or display ELF file information. There are also some ancillary components that I've never modified. I believe at some point it may also have contained compilers, however these have long since moved to their own separate source base.

The Link-Editor

When you mention the link-editor, most folks think of ld(1). You'll find this under sgs/ld. However, this binary is only a stub that provides argument processing and then dynamically loads the heart of the link-editor, libld.so. This library provides two flavors, a 32-bit version, and a 64-bit version, both capable of producing a 32-bit or 64-bit output file. The class of library that is loaded, is chosen from the class of the first input relocatable object read from the command line. This model stems from a compiler requirement that the link-editor class remain consistent with various compiler subcomponents.

The Runtime Linker

However, there's another link-editor that is required to execute every application on Solaris. This editor takes over where the standard link-editor left off, and is referred to as the runtime-linker, ld.so.1(1). You can find this under sgs/rtld. The runtime linker takes an application from exec(2), loads any required dependencies, and binds the associated objects together with the information left from ld(1). The runtime linker can also be called upon by the application to load additional dependencies and locate symbols.

This very close association of ld(1) and ld.so.1(1), is one reason the link-editors are considered part of the core OS rather than a component of the compilers. This separation has also insured the link-editors are compiler neutral.

One historic area of the runtime linker is its AOUT support. Objects from our SunOS4.x release were in AOUT format, and to aid customer transition from this release to Solaris, support for executing AOUT applications was provided by ld.so.1(1). We keep thinking that we're long past this transition need, and that this support could be purged from the system. However, we continue to come across customers that are still running an AOUT binary on Solaris. Sometimes the customer is Sun!

Also, if you poke around the relocation files for ld(1) and ld.so.1(1), you'll find a mechanism for sorting and counting relative relocations. This allows a faster processing loop for these relocations at runtime. Bryan did this before going on to bigger and better projects. It took him a couple of deltas to get things right, but he was a young lad back then.

Support Libraries

There are various support libraries employed by the link-editors. A debugging library, liblddbg.so, is employed by ld(1), ld.so.1(1) and elfdump(1) to provide tracing diagnostics. A common library is used to insure the debugging information looks consistent between the various tools. ld(1) uses libldmake.so to provide .make.state support, and libldstab.so for generic .stabs processing. ld.so.1(1) uses librtld.so for extending the runtime dynamic linking support, and librtld_db.so for mdb(1) and various proc tool support. ld.so.1(1) also used libld.so to process relocatable objects.

As you can see, there is a lot of interrelationships between the various components of the link-editors. The interfaces between these components are private and often change. When providing updates to the link-editors in patches and updates, this family of components is maintained and supplied as a whole unit.

Proto Build

As part of building the link-editor components, you might notice that we first build a version of ld(1) under sgs/proto, then use this version of ld(1) to build the other link-editor components, including the final ld(1). This two-stage build has developed as we frequently use new link-editor capabilities and flags to build our own components. A case of eating your own dog food. Without this two-stage build we would first have to integrate a version of ld(1) that provides the new capabilities, wait a week or two for this version of ld(1) to propagate into developers build environments, and then integrated the updates that require to use the new capabilities. Our two-stage build makes for a much faster turn-around. And, should we break something, we're usually the first to find out as we develop our changes.

Package Build

Under sgs/packages you'll see we have the capability of building our own package. This isn't the official package(s) that the link-editors are distributed under, but a sparse package, containing all our components. This package is how we install new link-editors quickly on a variety of test machines, or provide to other developers to test new capabilities or bug fixes, before we integrate into an official build. Note, there's no co-ordination between this package and the official package database, it's really no different than tar(1)'ing the bits onto your system, except you can back the changes out!

Patches

We make a lot of patches. Sure, there are bugs and escalations that need resolving, but we frequently have to make new capabilities available on older releases. The compilers are released asynchronously from the core OS, and new capabilities required by these compilers must be made available on every release the compilers are targeted to.

We have a unique way of generating patches. When asked to generate a patch we typically backport all the latest and greatest components. As I described earlier, there's a lot of interaction between the various components, and thus trying to evaluate whether an individual component can be delivered isn't always easy. So, we've cut this question out of the puzzle from the start, and always deliver all the link-editor components as a family.

Trying to isolate a particular bug fix can also be challenging. It may look like a two line code fix addresses a customer escalation, but these two lines are often dependent on some other fixes, in other files, that occurred many months before. Trying to remember, and test for all these possible interactions can be a nightmare, so we've removed this question from the puzzle too.

When we address a bug, we address it in the latest source base. If the bug can't be duplicated, then it may have been fixed by some previous change, in which case we'll point to the associated patch. Otherwise, we'll use all the resources available on the latest systems to track down and fix the issue. Yep, that means we get to use the latest mdb(1) features, dtrace(1M), etc. There's nothing more frustrating that having to evaluate a bug on an old release where none of your favorite tools exist. Sometime we have to fall back to an older release, but we try and avoid it if we can.

Having a fix for the issue, we'll integrate the changes in the latest Solaris release. And, after some soak time, in which the fix has gone through various test cycles and been deployed on our desktops and building servers, we'll integrate the same changes in all the patch gates. Effectively, we're only maintaining one set of bits across all releases. This greatly reduces the maintenance of the various patch environments, and frees up more time for future development.

This model hasn't been without some vocal opponents - "I want a fix for xyz, and you're giving me WHAT!". But most have come around to the simplicity and efficiency of the whole process. Is it flawless? No. Occasionally, regressions have occurred, although these are always in some area that has been outside of the scenarios we're aware of, or test for. Customers always do interesting things. But it will be a customer who finds such as issue, either in a major release, update or patch. Our answer is to respond immediately to any such issues. Our package build comes in very handy here.

You can always find the bugs we've fixed and patches generated from our SUNWonld-README file.

Other Stuff

Some other support tools are elfdump(1), ldd(1), and pvs(1). And, there's crle(1), moe(1), and lari(1). I quite enjoyed coming up with these latter names, but you might need some familiarity with old American culture to appreciate this. Which is rather odd in itself, as I'm British.

Anyway, hopefully this blog has enlightened you on navigating the OpenSolaris hierarchy in regard the link-editors.

Have fun, and in respect for a popular, current media event - may the source be with you.

Comments

guest — Friday June 17, 2005

Sergey — Friday June 24, 2005

Rod Evans — Friday June 24, 2005

The source code that should be under sgs/rtld is where you'd find the implementation of the dl\* family of routines.

However, just before launch, there was an issue with the build that resulted in the runtime linker code being mistakenly omitted. I'm told it will appear with the next OpenSolaris delivery.

Sorry for the confusion.

Published Elsewhere

[15] same name, different directories

Blog Index (rie)

[17] reducing dlsym() overhead