recent improvements to KDC database robustness

[Tom Yu]

We recently made two sets of commits to the master branch of the krb5
source tree that improve the robustness of the Berkeley DB btree ("DB2")
KDC database back end against rare instances of database corruption.
These improvements, which will be available in upcoming releases, are
recursive dump support and bug fixes for cross-endian access.

We have restored the recursive database dump capability for the DB2 back
end.  This will help sites recover from database corruption due to power
failure or program crash.  The typical symptoms are that some valid
principal records become inaccessible to random access (e.g., ordinary
KDC operation), sequential access (e.g., full database dumps), or
(rarely) both.  Restoration of the recursive dump capability is part of
the upcoming krb5-1.15 release.

We also fixed bugs that could corrupt databases during cross-endian
access.  This is when a database file that was created on a platform of
one byte order is accessed on a platform of the opposite byte order.  It
is generally better to dump and reload the database when moving to a
platform of a different byte order, but the DB2 code does (in principle)
support cross-endian access.  We expect deployments with cross-endian
database access to be rare, but these bug fixes will be in the upcoming
krb5-1.14.x and krb5-1.13.x patch releases, in addition to the krb5-1.15
release.

We do not fully understand the mechanisms that would produce the
symptoms of the few instances of KDC database corruption reported in the
field.  In the future, we may create tools to help us analyze instances
of such corruption.  If you have experienced corruption of a DB2 KDC
database, it would be helpful if you could preserve a copy of the
corrupted database for future analysis by such tools.

We are also in the early stages of evaluating LMDB as a considerably
more robust replacement for Berkeley DB as a KDC database back end:

    https://k5wiki.kerberos.org/wiki/Projects/LMDB_KDB

We welcome your feedback on the DB2 changes and the LMDB proposal.

Thanks,
-Tom