RFC: preauth benchmarking methodology

[Dmitri Pal]

On 06/03/2011 03:50 PM, Nathaniel McCallum wrote:
> On Fri, 2011-06-03 at 15:48 -0400, Marcus Watts wrote:
>>> Date:    Fri, 03 Jun 2011 15:07:16 EDT
>>> To:      krbdev at mit.edu
>>> From:    Nathaniel McCallum <npmccallum at redhat.com>
>>> Subject: RFC: preauth benchmarking methodology
>>>
>>> All the code referenced below comes from here:
>>> https://github.com/npmccallum/krb5-anonsvn/tree/perftest/src/plugins/preauth/pe
>>> rftest
>>>
>>> As part of the FreeIPA project (http://freeipa.org) we are attempting to
>>> add support for a variety of preauth mechanisms, such as yubikey, rsa,
>>> and others.  One of the major concerns that has come up in our testing
>>> is that while the current krb5 preauth mechanisms are quite quick to
>>> verify, the use of external services like yubikey which may introduce
>>> multi-second delays introduces scalability problems due to the
>>> non-threaded, synchronous main-loop of krb5.
>>>
>>> Before we attempt to fix the problem however, we need to make sure that
>>> we have a standardized testing suite to measure our progress.  This
>>> suite should be reusable for krb5 in other ways as well.
>>>
>>> The basic idea is that we need to simulate reproducible delays in a
>>> preauth plugin and measure the total responsiveness of the server when
>>> these delays appear.  To this end I've created a preauth plugin called
>>> 'perftest' which always approves the preauth after a delay.  The delay
>>> is controlled by the name of the principal, where 1 at REALM.COM would
>>> delay for 1 millisecond and 3000 at REALM.COM would delay for 3 seconds.
>>>
>>> Then we measure the speeds of a set of kinit's by using the simulexec.py
>>> file (in the repo above) which will execute a set of commands with a
>>> given concurrency and repetition values.  The output of the script is
>>> CSVs with the columns:
>>>   1. Total size
>>>   2. Total time (seconds)
>>>   3. Parallelism
>>>   4. Successes
>>>   5. Failures
>>>   6. Average time of successes (seconds)
>>>   7. Standard deviation of #4
>>>
>>> We will of course need to use a parallelism greater than the number of
>>> worker processes in the kdc, or the test will be useless.
>>>
>>> Does anyone have any further thoughts?
>>>
>>> Nathaniel
>>>
>>> _______________________________________________
>>> krbdev mailing list             krbdev at mit.edu
>>> https://mailman.mit.edu/mailman/listinfo/krbdev
>> Is your krb5kdc single threaded when it does a preauth choice?
> Yes. The goal is to move to an asynchronous event loop and measure the
> difference in performance.
>
>> Older versions of krb5kdc definitely were.  If a preauth method
>> took 2 seconds to execute, that's 2 seconds during which any
>> other request sent in would block -- and perhaps be redirected
>> to another kdc depending on how patient the client was.
> That is my assumption as well.
>
> Nathaniel
>
> _______________________________________________
> krbdev mailing list             krbdev at mit.edu
> https://mailman.mit.edu/mailman/listinfo/krbdev
>
>
Probably we should test with multiple worker processes.
So this is another dimension to throw into the result matrix.

-- 
Thank you,
Dmitri Pal

Sr. Engineering Manager IPA project,
Red Hat Inc.


-------------------------------
Looking to carve out IT costs?
www.redhat.com/carveoutcosts/