My Relocations Don't Fit - Position Independence

Rod Evans — Tuesday April 26, 2005

Surfing with the Linker-Aliens

A couple of folks have come across the following relocation error when running their applications on AMD64:

  $ prog
  ld.so.1: prog: fatal: relocation error: R_AMD64_32: file \\
      libfoo.so.1: symbol (unknown): value 0xfffffd7fff0cd457 does not fit

The culprit, libfoo.so.1 has been built using position dependent code (often referred to as non-pic).

Shared objects are typically built using position independent code, using compiler options such as -Kpic. This position independence allows the code to execute efficiently at a different address in each process that uses the code.

If a shared object is built from position-dependent code, the text segment can require modification at runtime. This modification allows relocatable references to be assigned to the location that the object has been loaded. The relocation of the text segment requires the segment to be remapped as writable. This modification requires a swap space reservation, and results in a private copy of the text segment for the process. The text segment is no longer sharable between multiple processes. Position-dependent code typically requires more runtime relocations than the corresponding position-independent code. Overall, the overhead of processing text relocations can cause serious performance degradation.

When a shared object is built from position-independent code, relocatable references are generated as indirections through data in the shared object's data segment. The code within the text segment requires no modification. All relocation updates are applied to corresponding entries within the data segment.

The runtime linker attempts to handle text relocations should these relocations exist. However, some relocations can not be satisfied at runtime.

The AMD64 position-dependent code sequence typically generates code which can only be loaded into the lower 32-bits of memory. The upper 32-bits of any address must all be zeros. Since shared objects are typically loaded at the top of memory, the upper 32-bits of an address are required. Position-dependent code within an AMD64 shared object is therefore insufficient to cope with relocation requirements. Use of such code within a shared object can result in runtime relocation errors cited above.

This situation differs from the default ABS64 mode that is used for 64-bit SPARCV9 code. This position-dependent code is typically compatible with the full 64-bit address range. Thus, position-dependent code sequences can exist within SPARCV9 shared objects. Use of either the ABS32 mode, or ABS44 mode for 64-bit SPARCV9 code, can still result in relocations that can not be resolved at runtime. However, each of these modes require the runtime linker to relocate the text segment.

Build all your shared objects using position independent code.


Update - Wednesday March 21, 2007

If you believe you have compiled all the components of your shared object using -Kpic, and still see an error of this sort, look a little more closely. First, determine if the link-editor thinks the shared object contains text relocations.

  $ elfdump -d library | fgrep TEXTREL

If this flag is found, then the link-editor thinks this file contains non-pic code. One explanation might be that you have include an assembler file in the shared library. Any assembler must be written using position-independent instructions. Another explanation is that you might have included objects from an archive library as part of the link-edit. Typically, archives are built with non-pic objects.

You can track down the culprit from the link-editors debugging capabilities. Build your shared object.

  $ LD_OPTIONS=-Dreloc,detail cc -o library .... 2> dbg

The diagnostic output in dbg can be inspected to locate the non-pic relocation, and from this you can trace back to the input file that supplied the relocation as part of building your library.


Update - Wednesday April 14, 2010

A question arose on how to interpret the diagnostic output so as to determine which input file is non-pic. You can look for the "creating output relocations" information. These are the relocation records that are created in the output file, and must be processed at runtime. The non-pic relocations will probably be against the .text section. So, if you have:

  debug: creating output relocations
  debug:          type                       offset     addend  section        symbol
  debug:        R_SPARC_HI22                  0x4d4          0  .SUNW_reloc    foo1

Then you can associate the offset with an output section:

  % elfdump -cN.text output-file.so 

  Section Header[7]:  sh_name: .text
    sh_addr:      0x4c0           sh_flags:   [ SHF_ALLOC SHF_EXECINSTR ]
    sh_size:      0x28            sh_type:    [ SHT_PROGBITS ]
    sh_offset:    0x4c0           sh_entsize: 0
    ...

Another aid might be to also use the link-editors -znocombreloc option. This suppresses the normal combination of output relocations sections, and might provide for a more informative diagnostic:

  debug: creating output relocations
  debug:          type                       offset     addend  section        symbol
  debug:        R_SPARC_HI22                  0x4d4          0  .rela.text     foo1

Here you can now see that the relocation is against the .text section.

Having found a non-pic relocation, search back in the diagnostics and try and find the matching input relocation using the relocation type and symbol name. You should find something like:

  debug: collecting input relocations: section=.text, file=foo.o
  debug:          type                       offset     addend  section        symbol
  debug:     in R_SPARC_HI22                   0x14          0  [13].rela.text foo1
  debug:    out R_SPARC_HI22                   0x14          0  .text          foo1

Here, the input relocation is against the text section (.rela.text), and to provide for this, an output relocation must be produced against the .text section. The non-pic culprit is the file foo.o.

You might be able to discover your non-pic relocations by just scanning through the "collecting input relocations" information. But for large links this can be a substantial amount of information to digest.

Surfing with the Linker-Aliens

Comments

Neil Martin — Saturday April 12, 2008

This helped me resolve an issue with apache2.2.4,perl5.8.8,mod_perl2.0.3 with solaris 10 x86 /amd64 using studio 12 when installing mod_perl . The problem turned out to be in the libperl.a archive, though everything in it appeared to be PIC, I believe globals.o was somehow not compiled with the PIC option. After removing that object unresolved symbols starting showing. Recompiling globals with -m64 and -KPIC solved the problem.

Joe Ganley — Friday May 28, 2010

Hmm. I have this problem - I get the "does not fit" error, and elfdump shows the TEXTREL object, but the linker debug output doesn't contain any "creating output relocations."

Rod — Friday May 28, 2010

You set "LD_OPTIONS=-Dreloc,detail" and captured the standard error, right? What relocation titles did you see in the output?

Joe Ganley — Friday May 28, 2010

Lots of "collecting input relocations", each with many lines of type/offset/addend/section/symbol. Sections are all .text and .rela.text, in matched pairs. That's all the debug output contains.

Joe Ganley — Friday May 28, 2010

Never mind. The sorry version of vi on my Solaris machine was truncating the debug file. It does in fact contain output relocations. Thanks for this post; I'm sure it will now help me nail down the issue.

David Kirkby — Monday August 30, 2010

On my system at least, the following two files in the /usr file system shows an output with:

$ elfdump -d library | fgrep TEXTREL

\* /usr/lib/amd64/libcanberra.so
\* /usr/lib/amd64/libglib-2.0.so

Is this process applicable to SPARC?

Should 32-bit shared libraries not show anything with the following command?

$ elfdump -d library | fgrep TEXTREL

What can I do when
\* I know there's no assembler
\* Compiling code, while setting

CC="gcc -m64 -fPIC"

still results in this. So I'm sure -fPIC is compiling the code.

Is there anything other than assembler and compiling without -fPIC/-KPIC that can cause this?

David Kirkby

Rod — Monday August 30, 2010

Well built 32-bit and 64-bit objects should \*not\* have relocations against read-only sections (ie. the TEXTREL test should reveal nothing).

Looking at a SPARC machine I see these same objects have text relations too.

First, I'd try and identify what the text relocations reference. For example, if I look at the first relocation:

% elfdump -r /usr/lib/sparcv9/libglib-2.0.so
....
type offset addend section symbol
R_SPARC_RELATIVE 0xc89b8 0xc02d0 .SUNW_reloc

it requires to update the offset at "0xc02d0".
Now if I look at what section this is I see:

% elfdump -c /usr/lib/sparcv9/libglib-2.0.so
....
Section Header[13]: sh_name: .rodata1
sh_addr: 0xbb290 sh_flags: [ SHF_ALLOC ]
sh_size: 0xce16 sh_type: [ SHT_PROGBITS ]

So, it looks like a read-only data item (const) needs relocating. This can happen if you have a table of addresses, and the table is defined const.

The studio compilers recognize this condition and normally place the table in a .picdata section which is given write access, and gets places in the data segment. I see this object has a .picdata section too.

What I'd do next is run your link-edit with LD_OPTIONS=-Dreloc,detail and trace back where the "0xc02d0" offset originates from (ie, which input file requires this relocation).

David Kirkby — Saturday September 18, 2010

I'm still puzzled. I'm trying to compile the Lisp interpreter 'ECL'.

http://ecls.sourceforge.net/resources.html
(latest source is git clone git://ecls.git.sourceforge.net/gitroot/ecls/ecl
)

I'm doing this 32-bit on a Xeon processor as that's the fastest machine I've got, and requires no messing around with CFLAGS, but I get the same issues on SPARC or on 64-bit builds.

I compiled with the debug options you suggest: I see in the output locations section:

debug: creating output relocations
debug: type offset section symbol
debug: R_386_JMP_SLOT 0x806cfc8 .rel.plt atexit
debug: R_386_JMP_SLOT 0x806cfcc .rel.plt __fpstart
debug: R_386_JMP_SLOT 0x806cfd0 .rel.plt exit
debug: R_386_JMP_SLOT 0x806cfd4 .rel.plt _exit
debug: R_386_JMP_SLOT 0x806cfdc .rel.plt __deregister_frame_info_bases
debug: R_386_JMP_SLOT 0x806cfe4 .rel.plt __register_frame_info_bases
debug: R_386_JMP_SLOT 0x806cff8 .rel.plt fprintf
debug: R_386_JMP_SLOT 0x806cffc .rel.plt printf
debug: R_386_JMP_SLOT 0x806d008 .rel.plt __filbuf
debug: R_386_JMP_SLOT 0x806d014 .rel.plt ungetc
and thousands of similar lines.

Then using the options -cN.text to elfdump I see:

drkirkby@hawk:~/ecl$ elfdump -cN.text ./build/libecl.so

Section Header[13]: sh_name: .text
sh_addr: 0x37540 sh_flags: [ SHF_ALLOC SHF_EXECINSTR ]
sh_size: 0x11bc0c sh_type: [ SHT_PROGBITS ]
sh_offset: 0x37540 sh_entsize: 0
sh_link: 0 sh_info: 0
sh_addralign: 0x10
drkirkby@hawk:~/ecl$

I don't understand how I'm supposed to connect the offsets from the first set of data with what's in the second, to allow the suspect function to be identified.

Rodrick Evans — Saturday September 18, 2010

The R_386_JMP_SLOT relocations are fine. This is how pic code resolves a function all. You want to looks at relocations against the .text section.

Try to follow through a simple example. Compile some code without a PIC flag:

% cat foo.c
extern int bar();
int foo() { return (bar()); }
% cc -c foo.c
% elfdump -r foo.o

Relocation Section: .rela.text
type offset addend section symbol
R_SPARC_WDISP30 0x14 0 .rela.text bar

Now follow this relocation through a link-edit that builds a shared object:

% LD_OPTIONS=-Dreloc,detail ld -z nocombreloc -G foo.o
...
debug: collecting input relocations: section=.text, file=foo.o
debug: type offset addend section symbol
debug: in R_SPARC_WDISP30 0x14 0 [13].rela.text bar
debug: out R_SPARC_WDISP30 0x14 0 .text bar

This indicates that ld() is going to generate a relocation against the .text section:

debug: creating output relocations
debug: type offset addend section symbol
debug: R_SPARC_WDISP30 0x404 0 .rela.text bar

Which you can see in the output file:

% elfdump -r a.out

Relocation Section: .rela.text
type offset addend section symbol
R_SPARC_WDISP30 0x404 0 .rela.text bar

It's this relocation against the text section (.rela against the .text) that is resulting in the flag:

% elfdump -d a.out | fgrep TEXT
[11] TEXTREL 0
[16] FLAGS 0x4 [ TEXTREL ]

David Kirkby — Sunday October 24, 2010

Em, I'm still stuck on sorting out the problem with the ECL Lisp interpreter.

I certainly have a text relocation problem:

drkirkby@hawk:~/sage-4.6.rc0/local/lib$ elfdump -d libecl.so | grep TEXTREL
[24] TEXTREL 0
[33] FLAGS 0x4 [ TEXTREL ]

drkirkby@hawk:~/sage-4.6.rc0/local/lib$ elfdump -cN.text libecl.so

Section Header[13]: sh_name: .text
sh_addr: 0x30c30 sh_flags: [ SHF_ALLOC SHF_EXECINSTR ]
sh_size: 0x105bdc sh_type: [ SHT_PROGBITS ]
sh_offset: 0x30c30 sh_entsize: 0
sh_link: 0 sh_info: 0
sh_addralign: 0x10
drkirkby@hawk:~/sage-4.6.rc0/local/lib$

But I can't seem to find anything in the output of elfdump -r

drkirkby@hawk:~/sage-4.6.rc0/local/lib$ elfdump -r libecl.so | grep Section
Relocation Section: .SUNW_reloc
Relocation Section: .rel.plt
drkirkby@hawk:~/sage-4.6.rc0/local/lib$ elfdump -r libecl.so | grep text
drkirkby@hawk:~/sage-4.6.rc0/local/lib$

So I'm puzzled.

I built this with degbug information using LD_OPTIONS and ended up with a 150 MB file. It's hard to navigate that.

Dave

Rod — Sunday October 24, 2010

If you used the "-z nocombreloc" flag you wouldn't get a single .SUNW_reloc section, but would get individual .rel sections associated to the output sections that must be relocated.

With these individual .rel sections, you might see a .rel.text, or perhaps .rel.rodata section. These would be the relocations that are causing TEXTREL. With these individual sections it should be easier to look at the debug output and associate their creation back to the input files that produce them.

David Kirkby — Sunday October 24, 2010

Thank you. I found that '-z nocombreloc' was not supported on one of the SPARC systems I looked at - I got a message about an unrecognisded option. But '-z nocombreloc' is supported on the x64 system I'm using.

I actually created a script 'inputloc' which showed only the section I \*think\* I need:

#!/bin/sh
if [ $# != 1 ] ; then
echo "Usage $0 objectfile" 2>&1
echo " Prints the section 'debug: collecting input relocations: section=.text'" 2>&1
exit 1
fi
LD_OPTIONS=-Dreloc,detail ld -z nocombreloc -G $1 2>&1 | sed -n '/debug: collecting input relocations: section=.text/,/debug: collecting/p'

but found that was outputing things on lots of files.

drkirkby@hawk:~/sage-4.6.rc0/spkg/build/ecl-10.2.1.p3/src/build/ext$ inputloc bytecmp.o | more
debug: collecting input relocations: section=.text, file=bytecmp.o
debug: type offset section symbol
debug: in R_386_GOTPC 0xd [2].rel.text _GLOBAL_OFFSET_TABLE_
debug: act R_386_GOTPC 0xd .text _GLOBAL_OFFSET_TABLE_
debug: in R_386_GOT32 0x1f [2].rel.text cl_env_p
debug: out R_386_GLOB_DAT 0x1f .got cl_env_p
debug: act R_386_GOT32 0x1f .text cl_env_p

I don't know if that's a bad sign or not. I got the feeling from what you posted above that I should not have these.

Rod — Sunday October 24, 2010

If -z nocombreloc isn't supported, then you have a rather old ld(1), and it wouldn't have been producing a combined relocation section (.SUNW_reloc). When we made .SUNW_reloc generation the default we added the -z argument too.

In the relocation diagnostics, it's the "out" relocations that are of interest:

debug: out R_386_GLOB_DAT 0x1f .got cl_env_p

However, the above relocation against the ".got" section shouldn't be a problem. As the .got section should be writable.

If you look at the sections within your output file (elfdump -c) the sections that are read-only (ie. don't have the SHF_WRITE sh_flags set) are the sections that should \*not\* have relocations against them.

The most common culprits of text relocations are against .text, .rodata or .rodata1. But maybe you have some other sections that are causing the error.

Look for the "out" diagnostics that are against any read-only section.

David Kirkby — Thursday October 28, 2010

Thank you.

Yes, one of my versions of 'ld' is old, but I'm using my OpenSolaris machine for debugging this, as it's a lot quicker than any of my SPARCs. Hence the 'ld' issue is not a problem.

I set:
LD_OPTIONS=-Dreloc,detail -z nocombreloc

built the library, then run

$ elfdump -c libecl.so

and get 52 sections.

Section 11 is interesting:

Section Header[11]: sh_name: .rel.got
sh_addr: 0x216c4 sh_flags: [ SHF_ALLOC SHF_INFO_LINK ]
sh_size: 0x2c0 sh_type: [ SHT_REL ]
sh_offset: 0x216c4 sh_entsize: 0x8 (88 entries)
sh_link: 5 sh_info: 26
sh_addralign: 0x4

as that appears to be a "got" section, but without any SHF_WRITE. Is that to be expected - I thought you were implying that the got sections should be writable.

If I understand correctly, the first section where relocations would be permitted is this one, which is another "got" section.

Section Header[26]: sh_name: .got
sh_addr: 0x170dbc sh_flags: [ SHF_WRITE SHF_ALLOC ]
sh_size: 0x1228 sh_type: [ SHT_PROGBITS ]
sh_offset: 0x160dbc sh_entsize: 0x4 (1162 entries)
sh_link: 0 sh_info: 0
sh_addralign: 0x4

as that has the SHF_WRITE flag.

So it looks like I need to concern myself only with sections 1-25 and sections 39-52, as none of them have SHF_WRITE, whereas sections 26 to 38 are writable.

Section Header[1]: sh_name: .SUNW_cap
Section Header[2]: sh_name: .eh_frame_hdr
Section Header[3]: sh_name: .hash
Section Header[4]: sh_name: .SUNW_ldynsym
Section Header[5]: sh_name: .dynsym
Section Header[6]: sh_name: .dynstr
Section Header[7]: sh_name: .SUNW_version
Section Header[8]: sh_name: .SUNW_versym
Section Header[9]: sh_name: .SUNW_dynsymsort
Section Header[10]: sh_name: .rel.text
Section Header[11]: sh_name: .rel.got
Section Header[12]: sh_name: .rel.data
Section Header[13]: sh_name: .rel.data.rel.ro.local
Section Header[14]: sh_name: .rel.data.rel
Section Header[15]: sh_name: .rel.data.rel.ro
Section Header[16]: sh_name: .rel.plt
Section Header[17]: sh_name: .plt
Section Header[18]: sh_name: .text
Section Header[19]: sh_name: .init
Section Header[20]: sh_name: .fini
Section Header[21]: sh_name: .rodata
Section Header[22]: sh_name: .rodata.str1.1
Section Header[23]: sh_name: .rodata.str1.4
Section Header[24]: sh_name: .rodata.cst8
Section Header[25]: sh_name: .rodata.cst4

and

Section Header[39]: sh_name: .symtab
Section Header[40]: sh_name: .strtab
Section Header[41]: sh_name: .comment
Section Header[42]: sh_name: .debug_abbrev
Section Header[43]: sh_name: .debug_info
Section Header[44]: sh_name: .debug_line
Section Header[45]: sh_name: .debug_frame
Section Header[46]: sh_name: .debug_loc
Section Header[47]: sh_name: .debug_pubnames
Section Header[48]: sh_name: .debug_pubtypes
Section Header[49]: sh_name: .debug_aranges
Section Header[50]: sh_name: .debug_ranges
Section Header[51]: sh_name: .debug_str
Section Header[52]: sh_name: .shstrtab

Before I go any further, am I right in thinking those sections above are the ones I should be concerned about? I'm puzzled by section 11, which is a "got" section, but not writable.

Dave

Rod — Thursday October 28, 2010

The ".rel." sections contain relocation records. These are the records that apply to the associated section, and indicate how offsets within that section must be updated. For example, the ".rel.got" contains the information that must be applied" to the ".got" section.

Therefore, the .rel section need only be readable. The section to which the relocation is applied is going to be written to. If this written to section doesn't have the SHF_WRITE, but the section has SHF_ALLOC, then it is part of the read-only memory image - hence you have a "text" relocation.

Sections 1-25 are all you need to be concerned about, as these are SHF_ALLOC but not SHF_WRITE.
Sections 39-52 can be ignored, as these don't have SHF_ALLOC, and won't become part of the memory image.

But the most glaring section would seem to be:

Section Header[18]: sh_name: .text

as there's also a:

Section Header[10]: sh_name: .rel.text

The latter instructs how to update the former, and this is probably the culprit. Find out from the debugging information which input file is contributing this .text section. This will be your non-pic code.

David Kirkby — Friday October 29, 2010

Thank you. Sections 10 and 18 are:

Section Header[10]: sh_name: .rel.text
sh_addr: 0x21638 sh_flags: [ SHF_ALLOC SHF_INFO_LINK ]
sh_size: 0x90 sh_type: [ SHT_REL ]
sh_offset: 0x21638 sh_entsize: 0x8 (18 entries)
sh_link: 5 sh_info: 18
sh_addralign: 0x4

Section Header[18]: sh_name: .text
sh_addr: 0x30c30 sh_flags: [ SHF_ALLOC SHF_EXECINSTR ]
sh_size: 0x105bdc sh_type: [ SHT_PROGBITS ]
sh_offset: 0x30c30 sh_entsize: 0
sh_link: 0 sh_info: 0
sh_addralign: 0x10

So it's section 18 I'm to be concerned about.

After capturing the output of the compilation (a 138 MB file with 1556648 lines), I see 13 sections marked "debug: creating output relocations" Fortunately, those make up a small part of the output, though there are still 10,000 lines.

These are here
http://boxen.math.washington.edu/home/kirkby/debug:-creating-output-relocations.txt

but most are irrelevant.

One or more sections are like this.

debug: creating output relocations
debug: type offset section symbol
debug: R_386_JMP_SLOT 0x5368c .rel.plt __deregister_frame_info_bases

<snip>

debug: R_386_RELATIVE 0x7fa7 .rel.text __get_exit_frame_monitor_ptr
debug: R_386_RELATIVE 0x7fb3 .rel.text __do_exit_code_ptr
debug: R_386_RELATIVE 0x8026 .rel.text __fsr_init_value_ptr
debug: R_386_RELATIVE 0x803f .rel.text trap_table

Is it those .rel.text sections I should be looking at in more detail? Looking at just the sections marked "debug: creating output relocations", I see the string .rel.text on just 180 lines

A couple of lines are

debug: R_386_RELATIVE 0x30c57 .rel.text __get_exit_frame_monitor_ptr
debug: R_386_RELATIVE 0x30c63 .rel.text __do_exit_code_ptr

but there are 180 in total

http://boxen.math.washington.edu/home/kirkby/output-relocation-sections-containing-rel.text.txt

If I understand correctly, I should be looking at those addresses and seeing where they map back to an input section, to locate the suspect file. But how do I use that address information?

"Section Header[18]: sh_name: .text" has 5 sets of numeric data, but I'm unsure how to use them, or the addresses in the file

http://boxen.math.washington.edu/home/kirkby/output-relocation-sections-containing-rel.text.txt

to actually get back to the relevant input section and so find the suspect code.

Dave

David Kirkby — Thursday February 10, 2011

At least part of this problem was resolved for me. The developer of the ECL interpreter said he had used "computed gotos" which are a GCC extension. It was them which were causing the problem, so they have been disabled on Solaris.

Stefano B — Monday February 21, 2011

Hi! I desperately need help! This post was the closest to my problem, I really hope you can help!!
I need to create a powerpc PIE executable, that is not a shared library, but a real executable linked as pie. This executable will be run by QEMU-USER.
I compile all with gcc -fPIC and then link with -Wl,-pie.
Checking all the linked object files with "readelf -d myobjectfile | fgrep TEXT" all seem to be PIC as nothing is produced by the previous comand.
Oddly giving the command "readelf -d mypie_executable | fgrep TEXT" to my final 'PIC' output executable i get: "0x00000016 (TEXTREL) 0x0"
Then if I try to load the file with QEMU-USER i get: "error while loading shared libraries: R_PPC_REL24 relocation at 0xb31f95a0 for symbol `__libc_start_main' out of range" !!!
Note that this happens with a simple hello world executable that does nothing: "int main(){ };"
During the linking phase something wrong happens.. I think that the linker does not expect I want an executable to be PIC ..
In fact this does not happen if I create a shared library, but I need an executable..
I am compiling and linking in Debian6 gcc 4.4.5 PowerPC.
"objdump -R myexecutable" gives:
Offset = 5a0 Type = R_PPC_REL24 Value=__libc_start_main
"readelf -r myexecutable" gives:
rela.dym at offset 0x39c
Offset=5a0 Info=40a Type=R_PPC_REL24 Sym.Value= 0000 Sym. Name+added = __libc_start_main
I tried using "LD_OPTIONS=-Dreloc,detail" but I do not get the same showed before.. I get "/\* Script for ld -pie: link position independent executable \*/ and then the script.. not very easy for me to understand (i can post it if it can help)
I really can't understand how it happens that from all PIC object files (if I checked all of them fine) linking with "pie" I get not a PIC executable!
Thank you very very much in advance!
Stefano B.

Rod — Monday February 21, 2011

Not being familiar with any PowerPC environments, I can only guess at
what might be occurring.

I suspect the reference to __libc_start_main is coming from one of the
crt files provided by the compiler driver. In other words, this reference
isn't from any of the files you are compiling -fPIC, but is from one of
the other files, added to the link, by the compiler.

Normally, non-pic references to external functions are translated into
procedure linkage table references (.plt) by the link-editor. But, for
this to occur, the link-editor must find a definition for the function,
and validate that the definition is defined as a function (the symbol
table entry should be defined as FUNC).

On a system close to me I see:

ubuntu: readelf -s /lib/libc.so.6 | fgrep __libc_start_main
2198: 00016c00 438 FUNC GLOBAL DEFAULT 12 __libc_start_main@@GLIBC_2.0

If you reference a libc like this during your final link-edit, the reference to __libc_start_main should be resolved to the definition in libc, and a .plt relocation would be created for runtime.

Hope this helps.

Stefano B — Monday February 21, 2011

Hi!
Thank you very much for your reply!
Sure that reference is coming from a crt object file added by gcc, I checked that file and it is PIC.

using readelf on my system:
Symbol table '.dynsym' contains 10 entries:
Num: Value Size Type Bind Vis Ndx Name

4: 00000000 0 FUNC GLOBAL DEFAULT UND __libc_start_main@GLIBC_2.0 (2)

Symbol table '.symtab' contains 71 entries:
Num: Value Size Type Bind Vis Ndx Name
60: 00000000 0 FUNC GLOBAL DEFAULT UND __libc_start_main@@GLIBC_

So it is recognized as a function.
readelf -s /lib/libc.so.6|grep __libc_start_main produces:
2320: 0001f720 224 FUNC GLOBAL DEFAULT 11 __libc_start_main@GLIBC_2.0

libc is linked by gcc (-lc), libc is regurarly PIC and there are no undefined symbols warnings/error .. though giving the command "readelf -d mypie_executable | fgrep TEXT" to my final 'PIC' output executable i get: "0x00000016 (TEXTREL) 0x0"

Thank you again!
Stefano B.

Rod — Monday February 21, 2011

Sorry, I can't explain what you're seeing, must be some subtlety with gcc/gld. The only comparable experiment I can do reveals __libc_start_main as a .plt reference.

ubuntu: cat > main.c
void main(){}
ubuntu: gcc main.c
ubuntu: readelf -r a.out
...
Relocation section '.rel.plt' at offset 0x264 contains 2 entries:
Offset Info Type Sym.Value Sym. Name
0804a00 00000207 R_386_JUMP_SLOT 00000000 __libc_start_main

Under Solaris you can set LD_OPTIONS-D<tokens> and discover how symbols are resolved and relocations built. I thought you could do this under gnu ld too, but I don't know the incantations.

Stefano B — Monday February 21, 2011

Hi!
I am also afraid it can be something not foreseen by who developed gnu gcc/ld .. a PIC executable is even not run by the shell.
I run them using qemu-user that can load them fine and go to the correct entry-point (well actually only the current developed version, not the last official release 0.13)..
I even do not have a Power-PC machine, I am running a Debian6 powerPC as a virtual machine in qemu-sytem.
I am just a student who sees this things (linkers, compilers, relocations..) for the very first time, really not easy..
I searched for LD_OPTIONS, all links are about Solaris, Sun ..
In my ld I found a -Map option.. I created one and I've uploaded the result into:
http://cid-9d0dd0f6c6e22fe8.office.live.com/browse.aspx/Public/Developing/master-degree-project?uc=1
together with dunps from objdump and readelf ..
I dunno, if you have time, for sure they will tell much more to you than to me :)
Thank you very much again!
Stefano B.

Rod — Tuesday February 22, 2011

Sorry, but I don't think there's anything I can offer.

Your site doesn't let me see any content, perhaps it's expecting me to login. But I'm not familiar with PPC, Qemu, and I only play a little in the gnu/gld environment.

On Solaris you can run pic, and non-pic code as an executable or shared object (although non-pic shared objects can have some address restrictions).

If you can't build a simple a.out with pic, and have it execute, you're going to have to call in some other experts.

Perhaps http://gcc.gnu.org/ml/gcc-help/?

Stefano B — Tuesday February 22, 2011

I see! Ok thank you very much anyway!
It is strange though that you can't open the files in my site.. it is my msn space, I can access fine without logging in with all other computers at my home, running different OSes, and firefox (I marked public the folder).. I didn't know where to post those little files, so I remembered I was given that space one day..
I've already asked help at gcc-help.. by the way they told me "readelf -d" is only meaningful for .so or executable, not .o .. and I should use "readelf -r" for checking relocs of those files.. I did, but I can't distinguish which are good, which say it is not PIC code ( the logs are in the Logs subfolder of that site ..).. hope somebody may help me :)
Best regards,
Stefano B.

rod — Tuesday February 22, 2011

I notice from my gnu testing, that an a.out is created with the following reference:

0804a004 00000207 R_386_JUMP_SLOT 00000000 __libc_start_main

whereas yours has a __libc_start_main@@GLIBC_ reference.

Perhaps there's something odd about how the versioned symbol __libc_start_main@@GLIB_2.0 is resolved from libc.so.6. A GNU compiler expert might have to unravel things from here.

Yon — Tuesday June 14, 2011

Quote: "If a shared object is built from position-dependent code, [...] the text segment is no longer sharable between multiple processes."

If the text segment is bigger than one memory page, does the system really make a private copy of the whole segment ? I mean, don't you think that the system only makes a private copy of pages that contain code lines concerned by relocation (Copy-On-Write feature) ?

rod — Tuesday June 14, 2011

Yes, only the pages that are written should become unsharable. However, there might be other overheads, like reserving swap space for the segment with the assumption that more pages are likely to be written.

guest — Thursday January 10, 2013

I am also facing the problem of relocations not fitting in. I am trying to build apache on solaris AMD64 platform. In apache if I disable mod_deflate, then the server works just fine. mod_deflate is the only module that is having problems.

I did what is suggested above, -Kpic -xmodel=medium, but still see the problem. Any other suggestions?

Surfing with the Linker-Aliens

Published Elsewhere

https://blogs.sun.com/rie/entry/my_relocations_don_t_fit/
https://blogs.oracle.com/rie/entry/my_relocations_don_t_fit/
https://blogs.oracle.com/rie/my-relocations-dont-fit-position-independence/

Surfing with the Linker-Aliens

[13] Loading Relocatable Objects at Runtime
Blog Index (rie)
[15] same name, different directories