problem with building libraries on AIX 5.3

[Matthew Andrews]

Hello,

  As previously mentioned on the kerberos at mit.edu mailing list, I(along
with Damian, who is cc:ed on this message but who does not subscribe, so
please keep him cc:ed on your replys) have been working on tracking down
the problems with building mit krb5 1.6 on AIX 5.3 with the IBM xlc
compiler, and have identified several issues.

  First it appears that the MIT kerberos build system is generating
rtl-able code when it generates shared libraries(config/shlib.conf
sets $LDCOMBINE to use the -brtl flag when linking libraries). On AIX
the convention is that libraries of this type have the .so extension.

Under aix if you wish to get the run-time linking behavior in an
executable it is also necessary to use the -brtl flag when linking the
executable(which is not specified as part of $CC_LINK_SHARED in
shlib.conf). Since AIX uses the .a extension for shared(but not run-time
linked) libraries, attempting to link an executable without the -brtl
flag when only *.so rtl style libraries will fail(the library will not
be looked for with the .so extension in this case.) and this is what has
been happening with kerberos 1.5 and later releases. I believe that in
the past this may not have been apparent as a problem because the
default behavior was to build static libraries.

The simple solution seems to be to add the -brtl flag to the
$CC_LINK_SHARED specification for aix, but this has the downside of also
requiring any other applications, or libraries which wish to use the MIT
krb5 libraries to use the run-time linker as well.

The libtool solution to this problem is to build a rtl-able version of
the library with the .so extension, and to build a dynamic(but not
shared) version of the library with the .a extension. This will allow
applications to link either using the "normal" AIX linking, or the AIX
run-time linker but I believe(but have not verified) that this means
that an application cannot get the normal AIX shared(but not run-time
linked) library behavior with a libtool generated library.

I think that the "right" answer is probably to generate run-time
linkable libraries with the .so extension, and then to create an archive
(.a) of that which can be used by applications which wish to do non-rtl
style linking, but I'm not sure if there's any good(read easy) way to
ensure that both the .a files, and the .so files get installed with the
build infrastructure that exists today(please if anyone has any ideas on
this, do share.)

  The next problem appears to be in the process to build the db2 kdb
plugin. for some reason on AIX(and on AIX only) the db2.so plugin is
linked against something called libdb.so. The plugin is also linked
against ALL of the component objects which make up this library, and it
doesn't appear to be being used anywhere else in the project so it's not
entirely clear what purpose it is being built for, let alone why the db2
plugin is linking against it.

  The third issue that we noticed is that the AIX libraries themselves
have a search path associated with them, and that this is not being set
explicitly by the MIT build system, and so it gets set based on the -L
arguments passed to the linker when building the library. I believe that
this should probably also be set to $PROG_RPATH just as it is in
executables.

I have attached a patch that should force all the krb5 applications to
be linked using the AIX run-time linker, removes libdb from the db2 kdb
plugin build line and adds $PROG_RPATH to the library search path when
building shared libraries.

also, for anyone looking for more info on AIX linking(woe be unto you)
here is a link to an IBM Redbook document that has pretty good
information on this:

http://www.redbooks.ibm.com/redbooks/SG245674/wwhelp/wwhimpl/js/html/wwhelp.htm

the sections on linking can be found in chapter 2.


-Matthew Andrews
-------------- next part --------------
A non-text attachment was scrubbed...
Name: krb.patch
Type: text/x-patch
Size: 2439 bytes
Desc: not available
Url : http://mailman.mit.edu/pipermail/krbdev/attachments/20070816/06fa51c3/attachment.bin