Settling An Old Score (Linker Division)
| Settling An Old Score (Linker Division) | |
Ali Bahrami — Wednesday June 14, 2006
For years, I worked on an interactive language used by scientists to do data analysis and visualization. That program makes heavy use of sharable libraries. Solaris was my primary development platform, due to its superior facilities for observing and debugging software, so my usual strategy was to write and debug my code under Solaris, and then move the results to the many other Unix (and VMS, Windows, and Macintosh) platforms that we supported.
I became very familiar with one quirk of the Solaris linker that bit me many times over the years. The issue has to do with how ld(1) handles the situation where it needs to replace an existing output file. Historically, this was handled by truncating the existing file and rewriting it in place. This preserves the existing inode, and any hard links that may happen to be pointing to it. However, it has a very bad effect on any running process that happens to be using that file. For example, if the output file is a sharable library, creating a new version while a program that uses it is running will inevitably cause that program to die in an unplanned and unexpected way.
If you're not a developer of software that runs for unbounded amounts of time, then you probably have not seen this behavior. If you develop code that does however, then you've almost certainly hit it at some point. In my case, I'd hit it about once a year, usually while multitasking, flipping back and forth between several xterms. Usually, it was obvious what had happened, and I'd quickly recover. Sometimes though, if I was debugging the program for some other reason, the unexpected SIGSEGV or SIGBUS would send me off into the weeds, debugging a mysterious problem in a part of the program unrelated to where I expected the problem to lie. After a minute or so, I'd realize what had happened.. Gack! Bitten again...
I cheerfully admit that this is not a big deal. However, I always wondered what reasons those people at Sun had for not changing it. I assumed that there was some subtlety that I was missing. Other platforms handle it in a different way, by having ld unlink the existing file first, and then create a new file under the same name. Any existing processes continue to see the old file, while the new file becomes available to new processes. When the last program with an open file descriptor exits, the Unix kernel removes the old file from the disk, in the standard Unix way.
I now work at Sun, on the Solaris linker and various related parts. One day the subject of this ld behavior came up, reminding me of my old questions. So, I started asking around. It turns out that no one at Sun is particularly fond of it either. The reasons for not having changed it boil down to the fact that it rarely causes problems, a desire to maximize compatibility with the past, and the fact that there are bigger things to worry about (almost everything).
The compatibility issue boils down to what happens if the output file has a link count that is greater than 1, as described earlier. It's rare for an output file to have multiple links, and even more rare for the makefile to not remove and relink all of the other names. Especially rare, since other operating systems (like Linux and the Macintosh) do break those links, and most Unix software is targetted at multiple operating systems (almost always including Linux).
As described in an earler blog entry, I've been using Solaris Zones to improve our linker testing. The basic approach I want to use for this is to replace the linker files in the test zone with symbolic links that point at the corresponding files in my development workspace. This means that every time I use make(1) to rebuild the linker components, the results will immediately become available in the zone. There is one problem with this idea: The way ld handles existing output files means that any processes using the old linker components will suddenly crash when I type make. Depending on what is running in the zone, this could destabilize and/or cripple the zone.
Of course, we could modify our makefiles to unlink the existing files before running ld, but this is tedious and error prone. I realized that it would be better to change the linker. At last, time to settle an old score.
The first step in making a change like this is to write and submit an ARC (Architectural Review Committee) case describing the change. We take backward compatibility very seriously here, so a change like this requires formal consideration and approval. I submitted PSARC 2006/353 a couple of weeks ago. It generated some discussion pro and con (the change causes existing hard links to be broken), but in the end, the undesirability of causing programs to die in uncontrolled ways, and the bonus of Linux compatibility won the day and the proposal was approved. I put back the code change earlier today, and it will appear in build 43, and soon after in OpenSolaris. Appropriately enough, more typing was involved in proposing the ARC case than in changing the code. The actual code change is very small.
It was a good learning experience for me to go through the ARC process, and gratifying to finally take my revenge on "my old friend". And of course, it will allow us to do better testing of the linker subsystem.
| [1] Testing...Without...System Into A Brick | [3] "Tentative" Symbols? |