performance of krb5 dejagnu test suite (tests/dejagnu)

[Ken Raeburn]

Until recently, our test suite in tests/dejagnu was set up to make  
about a dozen passes through the full set of tests, with each pass  
using a different configuration of encryption types or other  
parameters specified in the config files.  This is kind of wasteful,  
because some of the tests don't exercise anything new based on the  
encryption type beyond needing to be able to get tickets of a given  
encryption type, and one test would suffice for that.  One of the  
passes differed from another only in that TCP was used to communicate  
with the KDC rather than UDP; that won't affect, for example, how well  
the ftp client and server work, as long as tickets can be acquired,  
and a program like "kvno" can test that as well as the ftp client can,  
and with less extraneous stuff going on.

There are a few tests that depend on the session key type.  For  
example, telnet requires a DES key (and we should, for example, test  
that it'll get a DES session key as long as DES is an allowed  
encryption type, even if the default session key type is not DES, and  
even if we've already got credentials with other key types), and the  
appl/bsd and GSSAPI protocols have different paths followed for each  
of the early encryption types before we generalized them properly  
(assuming that we've actually done so even now).

I checked in some changes recently to make some of the tests get run  
only once during a full run, like for example the password-changing  
tests and the new kprop test.  I got rid of the TCP pass, and replaced  
it with a separate test exercising the TCP path, which only gets run  
once.  This cut down the run time of the test suite a bit, but it's  
still kind of long.

Eventually I think we should eliminate the multiple-pass stuff in the  
overall test suite configuration, and replace it with iterators over  
certain tests, describing different enctype configurations to use.   
For example, for the rlogin/rsh/rcp tests, only the session key we're  
going to get is relevant, not the master key type in the KDC or the  
key type of the TGS principal.  (The support in the works for  
encryption type negotiation -- RFC 4537 -- will complicate those tests  
that do depend on the subsession key encryption type too, as we'll  
have to make sure that the various possible changes in encryption type  
from session key to subsession key don't break anything.)  But we  
should also test that the KDC will run and we can get tickets if those  
key types are types not supported by the clients, as long as we have a  
compatible session key type.  And other tests may depend on whether  
krb4 support is enabled, or salt types used, etc, and the iterations  
over those tests should reflect that, not the full set of enctype  
configuration options we use now.

If anyone feels like doing a bit more speedup work, the slowest tests  
that still get run in every pass or nearly every pass include rsh,  
rcp, and rlogin; in a recent test on one of my reasonably fast  
machines, each invocation took ~9 seconds or more, totaling upwards of  
6.5 minutes (out of a total of nearly 11 minutes for that whole test  
suite).  As I said above, they do need to be run for different session  
key types, but not for variations in master key type and the like.

I suspect there are probably unnecessary delays somewhere in these  
tests, too.  If nothing else, there's a two-second delay put in to  
wait for the rshd/rlogind server to start up in standalone mode (and  
similarly in many of the other tests); testing for reachability in a  
loop, or having the server output something when it's started  
listening in standalone mode, would probably let us cut the delay  
significantly in normal cases, probably to a fraction of a second.   
That might not explain how some of the tests chew up 10-16 seconds  
though.

Ken