NSS for PKINIT, in-progress patches available, feedback sought

[Greg Hudson]

I built nss-3.12.11 and nspr 4.8.9 from source, built krb5 against it
(thanks for keeping this rebased against my recent preauth interface
work), and ran through the test suite.  I ran into the following
problem:

In crypto_get_pem_slot(), you call SECMOD_LoadUserModule with the module
spec "library=libnsspem.so".  There are several problems here.

1. Most importantly, if the module fails to load, SECMOD_LoadUserModule
still returns a module with a non-null function table, and then
SECMOD_UpdateSlotList crashes trying to access through the table.  I'm
not sure if this is an NSS bug or a caller bug.  The control flow is
that SECMOD_LoadModule calls secmod_LoadPKCS11Module to try to load the
library, gets an error return, goes to the "loser" label, and returns
the unloaded module.

2. libnsspem appears to be a Fedora special; that is, it doesn't appear
to exist in any Debian packages, googling for it finds only
Fedora-related links, there's no obvious project page for it, and it's
coded as a Mozilla source addon to be dumped in by the Fedora NSS SRPM.
I guess NSS is incapable of reading PEM files without this addon.  Since
our pkinit testing code uses PEM files, this seems to render the PKINIT
support untestable except on Fedora or RHEL (and also renders most of
our documentation of PKINIT valid on Fedora or RHEL, although it may be
possible to use PKINIT without PEM files).

3. Loading the module like this means that libnsspem.so needs to be
installed in the runtime linker path, unlike every other krb5 dependency
I'm aware of.

4. The name libnsspem.so is ELF-specific.

I guess none of these are real blockers if (1) should be treated as an
NSS bug, but they all seem unfortunate.