issue with krb5_prompter_posix() design

[Nicolas Williams]

On Thu, Apr 15, 2010 at 02:31:02PM -0700, Russ Allbery wrote:
> Nicolas Williams <Nicolas.Williams at oracle.com> writes:
> 
> > Yeah, yeah, whoever said that PAM is intended to be portable?  Ah, but
> > it has turned out to be used elsewhere, outside Solaris.  But even
> > ignoring the fact that Solaris makes it easier/more likely for apps to
> > work with a single allocator process-wide (and the fact that I think
> > that this is a Good Thing), I believe it's best to isolate allocator use
> > by library.  Meaning: that an API should provide its own deallocators
> > for all objects that it allocates, and should not deallocate objects it
> > did not allocate except through their corresponding deallocators (as
> > opposed to free()).  This principle is difficult to apply to prompters,
> > I agree, but hardly impossible.
> 
> I wholeheartedly agree.
> 
> This allocator/deallocator matching problem is often thought of as only a
> Windows issue, and hence not relevant to PAM, but I'm seeing more and more
> application software use its own allocators for one reason or another.
> (OpenLDAP, for example.)  In combination with dynamically loaded libraries
> and modules and various types of symbol binding, this can cause weird
> issues if allocations and deallocations cross library boundaries.

Note though: while I think good hygiene alone demands that allocation/
deallocation not cross API boundaries, it is a very, very, very, very,
very, very, very, VERY bad idea to make libraries use their own
allocators by default, and worse, much, much, much worse to _require_
that their callers tell them what allocators to use.  The same thing
applies to synchronization primitives.

It's an even worse ("worser") idea to make the choice of allocator
and/or synchronization primitives a global in the library.  OpenSSL, I'm
looking at _you_.

All library API designers should expect multiple distinct callers in one
process and aim to have zero global variables/state.  Any state that one
is tempted to think of as global should be encapsulated in a handle
(such as krb5_context).  Look at any major library that you can think
of, say, NSS, PKCS#11, OpenSSL, libevent -- all of them have got this
wrong, and some of them have thankfully evolved to address this (e.g.,
libevent).

If such design principles were heeded everywhere, and with sufficiently
advanced linker/loaders, much of DLL Hell (but not all of it[*]) could
be avoided.

[*] Sadly there are places where unintended global state can get
    trampled.  For example, POSIX file byte range locking, where closing
    _any_ open file descriptor referencing some file will drop all locks
    from that process, even those obtained by random libraries using
    different file descriptors for the same library.  OK, that's the
    worst, and most salient example.

Nico
--