malloc hang inside krb5_sendto_kdc

[Ken Weaverling]

I'm having some weird kerberos authentication issues since upgrading a 
Redhat box from 7.3 to RHEL 3. imap authenticates against a windows 2000 
kerberos server. That worked under 7.3 for well over a year on a fairly 
heavy loaded box (~300 imap connections open, a few new connects a 
second).

Since upgrading to RHEL 3, a few times a day an imap process will go 
into a CPU loop and consume all resources and sometimes other processes, 
such as our ldap server and apache server will hang until that imap 
process is killed.

Attaching to the processes always indicates the hang is within malloc() 
and always being called from krb5_sendto_kdc. The loop is somewhere 
within malloc. The function never returns. 

A sample backtrace...

#0  0xb735c164 in malloc_consolidate () from /lib/tls/libc.so.6 
#1  0xb735b769 in _int_malloc () from /lib/tls/libc.so.6 
#2  0xb735ab0d in malloc () from /lib/tls/libc.so.6 
#3  0xb75ad622 in krb5_sendto_kdc (context=0x1, message=0x81214a8, 
realm=0x1,  
    reply=0xbfffb510, use_master=1) at sendto_kdc.c:97 
#4  0xb75961f3 in send_as_request (context=0x8117ba0, request=
0xbfffb5d0,  
    time_now=0xbfffb510, ret_err_reply=0xbfffb594, ret_as_reply=
0xbfffb598,  
    use_master=1) at get_in_tkt.c:117 
#5  0xb7597420 in krb5_get_init_creds (context=0x8117ba0, creds=
0x81235dc,  
    client=0x811b9c8, prompter=0, prompter_data=0x0, start_time=0,  
    in_tkt_service=0x0, options=0x811b934,  
    gak_fct=0xb7597e20 <krb5_get_as_key_password>, gak_data=0xbfffc310,  
    use_master=1, as_reply=0xbfffb6bc) at get_in_tkt.c:946 
#6  0xb7598877 in krb5_get_init_creds_password (context=0x8117ba0,  
    creds=0x81235dc, client=0x811b9c8, password=0x8116b50 "", prompter=
0,  
    data=0x0, start_time=0, in_tkt_service=0x0, options=0x811b934) 
    at gic_pwd.c:156 
#7  0xb729d557 in pam_sm_authenticate () from /lib/security/pam_krb5.so 
#8  0xb75d2c06 in pam_fail_delay () from /lib/libpam.so.0 
#9  0xb75d2d81 in _pam_dispatch () from /lib/libpam.so.0 
#10 0xb75d4858 in pam_authenticate () from /lib/libpam.so.0 
#11 0x08062323 in server_input_wait () 
#12 0x0805bd55 in server_input_wait () 
#13 0x0805bfc6 in server_input_wait () 
#14 0x0805b225 in auth_plain_server () 
#15 0x08072b77 in mail_thread_compare_date () 
#16 0x080500c5 in ?? () 
#17 0x08103f2f in cmdbuf () 
#18 0x08056110 in fetch_rfc822_text () 
#19 0xb72ff748 in __libc_start_main () from /lib/tls/libc.so.6 
#20 0x0804c9f1 in ?? () 

Line 97 in sendto_kdc.c is:

94          for (i = 0; i < naddr; i++) 
95              socklist[i] = INVALID_SOCKET; 
96 
97          if (!(reply->data = malloc(krb5_max_dgram_size))) { 
98              krb5_xfree(addr); 
99              krb5_xfree(socklist); 
100             return ENOMEM; 
101         } 

This is krb5-libs-1.2.7-19 btw...

I do have one of four domain controllers running 2003 server, but 
krb5.conf points to the 2000 server. We tried pointing to 2003 server 
but it fails at times due to the tcp issue which I've read is fixed in 
1.3, which is why we aren't upgrading them all right now.

So .... is this a known bug? I've read some stuff that if a program 
clobbers malloc'ed memory it can sometimes exhibit a hang in 
_malloc_consolidate. 

Any hints on next steps I can take? I have an open support call with 
redhat for the past two weeks but it's not moving very quickly. :(

thx