krb5 and thread safety, revisited - do we need callbacks?

[Ken Raeburn]

About 18 months ago, there was some discussion on this list, and some 
in-person discussions among the MIT developers, concerning how to 
approach thread safety for the MIT Kerberos and GSSAPI libraries.  The 
proposal that I put forward (available at 
http://diswww.mit.edu:8008/menelaus.mit.edu/krb5dev/6761, though I 
think it might need tweaking still) was based on some assumptions:

* A platform may have multiple thread systems available.

* On such a system, the application should be able to use any one it 
wants.

* Our libraries need to use the same system API as the application.

* We don't want multiple libraries, built separately for every thread 
package.

So the proposal included a means for letting the application register 
callback functions through which the libraries could acquire and 
release locks, and manipulate thread-specific data.  These new 
interfaces would be added to each library, and each library would be 
expected to "notify" the libraries it depends on of the new callback 
functions, so that a GSSAPI application, for example, wouldn't need to 
know whether or not a Kerberos library was linked in.  (That's not a 
great example, unfortunately, because GSSAPI is a standardized API, so 
the application would need to know if it needed to use these extensions 
to the API or not.)  Thus, either libraries all the way up to the 
application need to provide such hooks in their APIs, or they become 
unusable with thread support functions other than the compiled-in 
defaults.

Recently, on the IETF SASL working group's mailing list, in a 
discussion of the SASL C API, some similar assumptions were called into 
question.  As Greg Hudson and others pointed out, on UNIX-like systems, 
the pthreads API has pretty much become the standard.  While there are 
other preemptive thread systems provided by some vendors, at least in 
some cases, they interoperate well enough with pthreads that it doesn't 
matter.  So, do we need to provide for the callback registration 
capability?  Or can we just go ahead and use pthreads on UNIX systems 
that support it, and whatever the basic Windows thread API is?

Mac OS 9, where there isn't a standard threading package, is no longer 
an issue, as we don't support it any more.  For other platforms without 
a standard threading package, we simply wouldn't do any locking.

Larry Greenfield brought up the matter of the GNU Pth library, which 
neither he nor I have first-hand experience with, but which was 
described as a popular cooperative threading library intercepting I/O 
syscalls that may block and switching threads at those points.  It's 
not clear to me how important that case is, if thread switching is only 
done at calls to certain well-defined library routines.

If we drop the callbacks, then we do have to decide what to do on 
platforms providing thread libraries.  If we're building UNIX shared 
libraries, it should be pretty easy -- we link our shared libraries 
against -lpthread or whatever, and the application writer doesn't need 
to worry about it.  If we're on a system where the C library provides 
no-op versions of the pthread functions, or the system supports weak 
external function references (i.e., if the function isn't defined, no 
link time error is reported, and the function's address is NULL), 
again, we simply use them (possibly with null function pointer checks), 
and if the application writer wants to use multiple threads, then the 
-lpthread versions will be pulled in and we'll get real locking.

Static libraries on a system without weak symbols or dummy versions in 
the C library would either require pulling in the pthread library, or 
would not have the locking calls.  I'm inclined to think that (1) we 
should be pushing towards shared libraries as the default, and (2) we 
should be trying to get developers to use the krb5-config script to get 
the list of libraries to link in.  So pulling in the pthread support 
doesn't seem like a big deal.

Is this an option?

Are there UNIX systems, or applications, where a preemptive threading 
system may be used that is not only not the pthread interfaces, but is 
not even compatible with it when used within a single program?  Does a 
cooperative threading package even matter, for these purposes?

How about any non-UNIX systems we might actually care about?  Any with 
multiple, incompatible native thread systems?  Are there incompatible 
non-native ones we should care about?  If they come up in the future, 
would it be good enough to require rebuilding the library in those odd 
cases?

As I've mentioned before, most of experience with threading systems is 
theoretical -- I've talked to people, and read a little, and thought 
about it a little.  I don't have any extensive experience to guide me 
in this.  If someone on this list does, please chime in...

Ken