kdc slowness recently - delayed login authentication

[Fletcher Cocquyt]

Hi,
Our development SULinux boxes are setup through PAM to authenticate first
against campus kerberos, then local machine accounts.
In the last 2-3 days users have noticed intermittent issues authenticating.
This morning I noticed 5-10 second delays after I hit enter on my password
before the auth attempt was finished against the kdc's and succeeded against
the local account - normally it takes less than a second.

Have there been any issues with the campus kerberos servers ?
Short of running a sniffer, how can I get more debug info considering this
problem is intermittent?

The problem seems to be characterized by more of these messages in the
system logs:
BEFORE:
/var/log/messages.4:Oct 13 09:24:07 irt-dev-cvs1 sshd(pam_unix)[20599]:
session closed for user jaxtell
/var/log/messages.4:Oct 13 09:24:07 irt-dev-cvs1 pam_afs[20610]: AFS
Authentication failed for user jaxtell. password was incorrect
/var/log/messages.4:Oct 13 09:24:07 irt-dev-cvs1 sshd(pam_unix)[20609]:
session opened for user jaxtell by (uid=0)
/var/log/messages.4:Oct 13 09:24:07 irt-dev-cvs1 pam_afs: AFS Authentication
failed for user jaxtell. password was incorrect
/var/log/messages.4:Oct 13 09:24:07 irt-dev-cvs1 sshd(pam_unix)[20609]:
session closed for user jaxtell
NOW:
/var/log/messages:Nov 13 16:28:40 irt-dev-cvs1 sshd(pam_unix)[13677]:
session closed for user jaxtell
/var/log/messages:Nov 13 16:28:40 irt-dev-cvs1 pam_afs[13688]: AFS
Authentication failed for user jaxtell. password was incorrect
/var/log/messages:Nov 13 16:28:40 irt-dev-cvs1 sshd(pam_unix)[13687]:
session opened for user jaxtell by (uid=0)
/var/log/messages:Nov 13 16:28:40 irt-dev-cvs1 pam_afs: AFS Authentication
failed for user jaxtell. password was incorrect
/var/log/messages:Nov 13 16:28:40 irt-dev-cvs1 sshd(pam_unix)[13687]:
session closed for user jaxtell
/var/log/messages:Nov 13 16:28:40 irt-dev-cvs1 pam_afs[13697]: AFS
Authentication failed for user jaxtell. password was incorrect
/var/log/messages:Nov 13 16:28:40 irt-dev-cvs1 sshd(pam_unix)[13696]:
session opened for user jaxtell by (uid=0)
/var/log/messages:Nov 13 16:28:40 irt-dev-cvs1 pam_afs: AFS Authentication
failed for user jaxtell. password was incorrect
/var/log/messages:Nov 13 16:28:40 irt-dev-cvs1 sshd(pam_unix)[13696]:
session closed for user jaxtell
/var/log/messages:Nov 13 16:28:40 irt-dev-cvs1 pam_afs[13707]: AFS
Authentication failed for user jaxtell. password was incorrect
/var/log/messages:Nov 13 16:28:40 irt-dev-cvs1 sshd(pam_unix)[13706]:
session opened for user jaxtell by (uid=0)
/var/log/messages:Nov 13 16:28:40 irt-dev-cvs1 pam_afs: AFS Authentication
failed for user jaxtell. password was incorrect

Are the auth servers overloaded at certain times of the day?

Thanks,
Fletcher.
fletch at med.stanford.edu