One cause of krb5 test suite slowness on some systems

[Greg Hudson]

On Fri, 2009-01-30 at 21:22 -0500, ghudson at MIT.EDU wrote:
> With that locally patched in my system's expect command, the
> tests/dejagnu part of the test suite dropped from 27 minutes to 12
> minutes, and the overall test suite runs in about 20 minutes.  That
> still probably reflects a lot of unnecessary delays (the "user" and
> "sys" times total to about 20 seconds) but it's progress.

Upon further investigation, I think the remaining twelve minutes clocked
by tests/dejagnu is almost accounted for from three sources:

1. We start daemons like kadmind and then run "tail -f" on a log file to
see when they have started.  On most systems, "tail -f" polls its input
once per second, so that's a one-second delay per system.  You can work
around this problem on Linux by running "tail -f -s 0.01" instead to
force faster polling.  This workaround reduced my runtime from 12m to
10m on my system, although that's pretty unscientific due to the next
item.

2. /dev/random exhaustion can result in unpredictable delays during the
test suite.  Hacking lib/crypto/prng.c to always use /dev/random reduced
my runtime from 10m to 6m.

3. When starting some daemons, we don't even have the ability to tail -f
a log file to look for a started message, so we have a hardcoded "sleep
2".  These can add up pretty fast--for instance, we start krshd six
times in each pass over rsh.exp, so in the 11 passes of the test suite,
that sleep alone consumes 132 seconds.  I haven't worked out a
comprehensive total, but I'm pretty confident that almost all of the
remaining six minutes is accounted for here.

None of these issues is trivially fixed in a clean, portable fashion,
but we have some ideas, and I believe Ken is working on some
improvements to #3 using t_inetd.