issue with krb5_prompter_posix() design

[Nicolas Williams]

On Wed, Apr 21, 2010 at 10:38:35AM -0400, Jeffrey Hutzelman wrote:
> --On Thursday, April 15, 2010 04:49:30 PM -0500 Nicolas Williams
> <Nicolas.Williams at oracle.com> wrote:
> >It's an even worse ("worser") idea to make the choice of allocator
> >and/or synchronization primitives a global in the library.  OpenSSL, I'm
> >looking at _you_.
> >
> >All library API designers should expect multiple distinct callers in one
> >process and aim to have zero global variables/state.  Any state that one
> >is tempted to think of as global should be encapsulated in a handle
> >(such as krb5_context).  Look at any major library that you can think
> >of, say, NSS, PKCS#11, OpenSSL, libevent -- all of them have got this
> >wrong, and some of them have thankfully evolved to address this (e.g.,
> >libevent).
> 
> All of what you say is true, for allocators and a variety of other
> cases. However, synchronization primitives are another beast
> entirely, because they generally don't work unless everything in the
> process agrees on what threading system is in use.  I know that
> Solaris provides more than one threads API, but that's not really
> more than one threading system, because they're just different
> interfaces to the same underlying primitives.  I work routinely with
> software (AFS) that is designed to support a threading model other
> than pthreads, and the right thing does _not_ happen if some
> libraries decide to use POSIX mutexes anyway.

I don't agree, for two different reasons.  First, ideally every OS
should provide a decent threading API to use, and the app (AFS in your
case) should just. use. that.  It seems that the existence of multiple
distinct, incompatible threading systems really reflects the [long and
tortuous] history of the adoption of threading in the Unix world; but
it's time to stop that and settle for pthreads.  Second, if the APIs
exported by the libraries I mentioned were designed such that there's a
caller context handle in all cases then there should be no need for
global state that requires synchronization, at least in _most_ cases.  I
can think of cases where global state must exist that also requires
synchronization, such as in the case of SQLite3's POSIX file byte range
locking first-close-drops-locks disaster workaround.  But for most of
the libraries I mentioned I believe that the problems lie in the API
designs, not in POSIX file byte range locking.

Incidentally, the two threading APIs in Solaris are compatible with each
other.  Also, threading ultimately gets intimately tied into libc -- it
has to be, because there are places where libc needs to do locking,
because of signals, fork() and other friends.  If you have a monolythic
application, you could realistically use your own threading library and
libc replacements (e.g., allocator) so that it all just works, but the
moment you mix in libraries that use the OS' threading system... you
lose.  Monolythic application development is sooo 1980, if not 1960.

And no, I don't think we could/should countenance the apparent answer
from OpenSSL and friends: that all libraries that need threading/
synchronization should be initialized by the application first, and
initialized with pointers to the threading/sync primitives that they
need.  The OpenSSL model means you can't layer libraries: no libkrb5 use
of OpenSSL, plugin frameworks must be radically altered, libsasl
couldn't have a plugin to use libgss, libldap couldn't use libsasl,
etcetera.  The world of Internet protocols that we live with happens to
layer many protocols over many others, and the only alternative to
layering libraries is to fork off helper processes to proxy for the
libraries that can't be layered-over (not necessarily a bad idea, but
certainly one not widely adopted, if at all).  And then, fork() can't
really be used by libraries, as it interferes with the application;
Solaris has forkx(2) to allow libraries to safely start child processes
that don't interfere with the application.

In other words: the simplest way out of this mess is to just use the
host OS' libc, allocator and threading API.  If that happens to suck for
some OS, complain to the vendor -- the workarounds people are foisting
on the rest of us are simply toxic.  (Yes, you could argue that if one
does not like OpenSSL then one should develop an alternative, and you'd
be right, on a level, but then, there's already N alternatives, all of
which got some of this wrong, and which already have enough developer
mindshare and momentum that it would be easier to work with their
developers than it would be to develop replacements and get them widely
accepted.)

Nico
--