krb5 thread support and excess support libraries -- seeking opinions, options

[Ken Raeburn]

On May 5, 2004, at 18:34, Nicolas Williams wrote:
> Yes, a new library does seem like the way to go.  Can you merge some
> utility libraries, perhaps?

The random archive libraries we use in the build process and don't 
install (pty, ss, db, apputils), I'm not worried about.  The ones we 
install currently seem to be: com_err, des425, gssapi_krb5, gssrpc, 
k5crypto, kadm5clnt, kadm5srv, kdb5, krb4, krb5, and now this new one 
tentatively named krb5support.

We could probably merge k5crypto and maybe des425 into krb5, the only 
concern being namespace pollution.  (Since we're starting to use export 
lists for UNIX shared libraries now, and plan to trim the list to as 
close to an "official" API as we can manage, that's actually an issue.  
If we had options like filter-object libraries on more platforms, we 
could offer different "views" of the combined library depending on what 
subset of the API was needed by the application, like standard GSSAPI 
versus our extended GSSAPI.  But we don't.)  They're already our two 
biggest libraries; the combination would far outweigh any other library 
we ship.  We would probably be able to drop a few more symbols from the 
export list -- anything in k5crypto used by krb5 and not intended for 
application use.

Maybe we could merge kdb5 into kadm5srv, though I think there are a 
couple of programs using kdb5 that don't use kadm5srv.

I'm not sure what else we could merge.

> Use lazy loading and the socket library won't get loaded into the
> com_err test program...

True, but I'd rather have the dependencies be more reasonable.  If the 
com_err code doesn't do any networking, and doesn't call any code that 
does networking, then the socket library shouldn't even be in the 
dependency graph.

For that matter, we should be able to link apps and libraries against 
just the libraries they use directly -- e.g., link against 
libgssapi_krb5 only, if it doesn't do any Kerberos stuff directly, and 
have the dependencies recorded in the library pull in the other needed 
libraries, preferably without messing with the symbol namespace, 
either.  But I don't think I've got time to investigate that soon for N 
platforms.

> I recommend a multi-pass build: first build/install the headers, then
> build/install the sources, ...

An option like that has been requested before, but making that the only 
way we do it makes it tricky to test the newly built code before 
installing it.  Instead, we'd have to build-and-partly-install in a 
different place, run tests, rebuild for a different location, and 
install.

It would simplify some things.  For example, our in-build-tree testing 
has to jump through some hoops setting LD_LIBRARY_PATH and _RLD_ROOT 
and other things based on the platform, to get the newly built 
libraries used instead of any installed ones that happen to be around.  
If we always installed before testing, we could drop a lot of those 
hacks.

But being able to test the executables you're about to install, as 
opposed to executables that in theory ought to be functionally 
identical to the ones you'll build and install later aside from some 
pathname changes, seems like a big win to me....

> OTOH, by now you're doing radical build surgery, you might want to
> consider nor moving other common code into the the thread-support
> library than you have to.

I'm having parsing your your sentence. :-)
Did you mean, not move any more code than I have to?  At the moment 
it's looking to me more like it should be a thread support library 
only.  I've tentatively moved the local-address code into libkrb5 as I 
indicated, and the getaddrinfo hacks are staying put (one massive 
header file with code, non-thread-safe caching done for Mac OS X only, 
no use of ) until I figure out what to do with them.

The surgery isn't that radical.  Most programs use a make variable 
shared throughout the entire tree to indicate the libraries they use.  
The com_err library's change from no library dependencies to having 
some is actually one of the bigger changes.  (We treat those two cases 
differently for constructing linker arguments.)

> What's cheaper?  More libraries?  Or more ignored (lazily loaded)
> dependencies in the resulting executables?

As for the location of the local-address lookup code and 
getaddrinfo/getnameinfo wrapper code, I don't think we want yet another 
new shared library for network stuff.  We're not going to support it 
for other random apps; using it from the KDC (maybe via the apputils 
lib) is an okay (not great) place to access "internal" routines.

If the TSD support code is the only stuff in the new support library 
I've already set up, it doesn't need to pull in any external libraries 
(except maybe system thread libraries on some unfortunate system, and 
of course the C library), so lazy loading doesn't make much difference.