RFC: preauth benchmarking methodology

[Marcus Watts]

> Date:    Fri, 03 Jun 2011 15:07:16 EDT
> To:      krbdev at mit.edu
> From:    Nathaniel McCallum <npmccallum at redhat.com>
> Subject: RFC: preauth benchmarking methodology
> 
> All the code referenced below comes from here:
> https://github.com/npmccallum/krb5-anonsvn/tree/perftest/src/plugins/preauth/pe
> rftest
> 
> As part of the FreeIPA project (http://freeipa.org) we are attempting to
> add support for a variety of preauth mechanisms, such as yubikey, rsa,
> and others.  One of the major concerns that has come up in our testing
> is that while the current krb5 preauth mechanisms are quite quick to
> verify, the use of external services like yubikey which may introduce
> multi-second delays introduces scalability problems due to the
> non-threaded, synchronous main-loop of krb5.
> 
> Before we attempt to fix the problem however, we need to make sure that
> we have a standardized testing suite to measure our progress.  This
> suite should be reusable for krb5 in other ways as well.
> 
> The basic idea is that we need to simulate reproducible delays in a
> preauth plugin and measure the total responsiveness of the server when
> these delays appear.  To this end I've created a preauth plugin called
> 'perftest' which always approves the preauth after a delay.  The delay
> is controlled by the name of the principal, where 1 at REALM.COM would
> delay for 1 millisecond and 3000 at REALM.COM would delay for 3 seconds.
> 
> Then we measure the speeds of a set of kinit's by using the simulexec.py
> file (in the repo above) which will execute a set of commands with a
> given concurrency and repetition values.  The output of the script is
> CSVs with the columns:
>   1. Total size
>   2. Total time (seconds)
>   3. Parallelism
>   4. Successes
>   5. Failures
>   6. Average time of successes (seconds)
>   7. Standard deviation of #4
> 
> We will of course need to use a parallelism greater than the number of
> worker processes in the kdc, or the test will be useless.
> 
> Does anyone have any further thoughts?
> 
> Nathaniel
> 
> _______________________________________________
> krbdev mailing list             krbdev at mit.edu
> https://mailman.mit.edu/mailman/listinfo/krbdev

Is your krb5kdc single threaded when it does a preauth choice?

Older versions of krb5kdc definitely were.  If a preauth method
took 2 seconds to execute, that's 2 seconds during which any
other request sent in would block -- and perhaps be redirected
to another kdc depending on how patient the client was.

						-Marcus Watts