LMDB KDB module design notes

[Nathaniel McCallum]

On Thu, Apr 12, 2018 at 12:34 PM, Greg Hudson <ghudson at mit.edu> wrote:
> On 04/12/2018 11:56 AM, Simo Sorce wrote:
>>> Transactions are per-environment, so if we use one database file and a
>>> write transaction for loads, loads would block the KDC.  That's worse
>>> than what we have with DB2.

Could loads be segmented into chunks to avoid blocking the KDC for the
entire operation?

>>> Alternatively we could load into a temporary database file and rename it
>>> into place like we do with DB2.  But we would then have to close and
>>> reopen the database between operations like we do with DB2, or somehow
>>> signal processes that have the database open to reopen it after a load
>>> completes.
>>
>> How common are loads ?
>
> That's hard to predict, but for a large database, having the KDC block
> for the lifetime of a load operation seems like a pretty noticeable problem.
>
>> As far as I know LMDB will let you keep reading during a transaction, so
>> the KDC would block only if there are write operations, but won't block
>> in general, right ?
>
> Yes.
>
>> The only write operations are on AS requests, when lockout are enabled
>> and when that triggers a change in the lockout fields. How common is that ?
>
> By default, every successful AS request on a principal requiring preauth
> updates the last_success timestamp.  If disable_last_success is set (but
> disable_lockout is not), only failed AS requests would cause a KDC write.

Could you create an opportunistic write queue for these ? This would
unblock the KDC during loads. The cost would be that the
aforementioned writes would not occur until the ends of the loads.
Slightly stale data is probably not a big deal for (at least)
last_success.

>> Would that something that can be mitigated by deferring those writes during
>> transactions ?
>
> I don't see a way in LMDB to check for a write transaction, or begin a
> write transaction without blocking.  Queueing those writes to be
> performed later would also add a lot of complexity.
> _______________________________________________
> krbdev mailing list             krbdev at mit.edu
> https://mailman.mit.edu/mailman/listinfo/krbdev