1.8.2 KDCs segfaulting on authdata processing
Mike Roszkowski
roszkowski at wisc.edu
Fri Sep 3 11:08:04 EDT 2010
Greg,
Thanks for the response. I made the changes you suggested last night
and was able to generate a couple of core files.
As you've guessed, I can't easily generate a request that causes
the problem.
I'll try to analyze the dumps today and see if I can get a
better idea of where the problem is.
Thanks for the help!
--Mike Roszkowski
Greg Hudson wrote:
> Sorry it's taken so long to get back to you on this. I'm moving
> discussion to a closed list (plus you, of course) since KDC crashes
> generally indicate a security issue.
>
> All processing of enc_tkt_reply->authorization_data goes through
> merge_authdata(). If you had a way of generating a request which caused
> a crash, you could set a breakpoint on merge_authdata and check the
> authdata array after each time.
>
> But it doesn't sound like you have that ability, so I think what you
> want to do is induce the crash to happen at the time the authdata array
> becomes invalid. I'd suggest the following temporary code modification:
>
> Index: kdc_authdata.c
> ===================================================================
> --- kdc_authdata.c (revision 24286)
> +++ kdc_authdata.c (working copy)
> @@ -502,8 +502,9 @@
> return 0;
>
> if (authdata != NULL) {
> - for (nadata = 0; authdata[nadata] != NULL; nadata++)
> - ;
> + for (nadata = 0; authdata[nadata] != NULL; nadata++) {
> + krb5_authdatatype t = authdata[nadata]->ad_type;
> + }
> }
>
> for (i = 0; in_authdata[i] != NULL; i++)
> @@ -548,6 +549,9 @@
> }
>
> *out_authdata = authdata;
> + for (i = 0; authdata[i]; i++) {
> + krb5_authdatatype t = authdata[i]->ad_type;
> + }
>
> return 0;
> }
>
> The first hunk should detect if the input authdata contains invalid
> pointers; the second hunk should verify that the resulting authdata
> array acontains valid pointers. You may need to compile with
> optimization turned off (configure with CFLAGS=-g) to make sure those
> dereferences don't get optimized out.
>
> On Fri, 2010-08-27 at 14:08 -0400, Mike Roszkowski wrote:
>> I've installed two 1.8.2 KDCs on RHEL5-64 that are supposed to replace our
>> current 1.4.x KDCs.
>>
>> When I switched them into production, the KDCs crashed pretty regularly
>> processing TGS requests from machines in our AD domain (running on
>> Windows Server 2008 R1). Our MIT KDCs handle authentication for the AD domain.
>> Everything works fine with the 1.4.x KDCs so I don't think there's any
>> problems with the domain trust, etc.
>>
>> The syslog entries for the crashes looked like this:
>>
>> Aug 3 06:17:35 gingersnap kernel: krb5kdc[7155]: segfault at 0000000000000005
>> rip 0000000000416786 rsp 00007fffb177fc60 error 4
>> Aug 3 06:49:01 gingersnap kernel: krb5kdc[12914]: segfault at 000000000028042e
>> rip 0000000000416786 rsp 00007fff15c9bab0 error 4
>> Aug 3 06:57:56 gingersnap kernel: krb5kdc[14948] general protection rip:416786
>> rsp:7fff525d3a40 error:0
>>
>> Aug 3 06:21:00 sugar kernel: krb5kdc[25955] general protection rip:416786
>> rsp:7fffd0e7a770 error:0
>> Aug 3 06:53:31 sugar kernel: krb5kdc[31689] general protection rip:416786
>> rsp:7fffb4f16a90 error:0
>>
>> I was able to get a crash dump and syslog showed the following:
>>
>> Aug 13 07:59:47 sugar kernel: krb5kdc[26180]: segfault at 0000000000000035 rip 0
>> 000000000416786 rsp 00007fff6e2026b0 error 4
>>
>> Here's the backtrace from the core file:
>>
>> (gdb) bt
>> #0 is_kdc_issued_authdatum (context=0x14fbde00, authdata=0x31,
>> desired_type=128) at kdc_authdata.c:411
>> #1 0x000000000041688a in has_kdc_issued_authdata (context=0x14fbde00,
>> authdata=0x14fd8290) at kdc_authdata.c:456
>> #2 only_pac_p (context=0x14fbde00, authdata=0x14fd8290) at kdc_authdata.c:1136
>> #3 0x0000000000417076 in handle_signedpath_authdata (context=0x14fbde00,
>> flags=<value optimized out>, client=0x7fff6e202a40, server=0x7fff6e202aa0,
>> krbtgt=<value optimized out>, client_key=<value optimized out>,
>> server_key=0x7fff6e202ca0, krbtgt_key=0x14fd8e20, req_pkt=0x7fff6e2048f0,
>> request=0x14fd7b90, for_user_princ=0x0, enc_tkt_request=0x14fd9620,
>> enc_tkt_reply=0x7fff6e202b00) at kdc_authdata.c:1186
>> #4 0x00000000004161c0 in handle_authdata (context=0x14fbde00, flags=176,
>> client=0x7fff6e202a40, server=0x7fff6e202aa0, krbtgt=0x7fff6e2029e0,
>> client_key=0x14fd7540, server_key=0x7fff6e202ca0, krbtgt_key=0x14fd8e20,
>> req_pkt=0x7fff6e2048f0, request=0x14fd7b90, for_user_princ=0x0,
>> enc_tkt_request=0x14fd9620, enc_tkt_reply=0x7fff6e202b00)
>> at kdc_authdata.c:784
>> #5 0x0000000000408c31 in process_tgs_req (pkt=0x7fff6e2048f0,
>> from=0x7fff6e204900, response=0x7fff6e204918) at do_tgs_req.c:744
>> #6 0x0000000000405523 in dispatch (pkt=0x7fff6e2048f0, from=0x7fff6e204900,
>> response=0x7fff6e204918) at dispatch.c:90
>> #7 0x0000000000415599 in process_packet (conn=<value optimized out>,
>> selflags=<value optimized out>) at network.c:1298
>> #8 0x00000000004148e7 in service_conn () at network.c:1638
>> #9 listen_and_process () at network.c:1729
>> #10 0x000000000041320d in main (argc=1, argv=0x7fff6e204b68) at main.c:939
>> (gdb)
>>
>> The problem appears to be that the enc_tkt_reply->authorization_data array
>> isn't terminated correctly:
>>
>> gdb) p *enc_tkt_reply->authorization_data[0]
>> $2 = {magic = -1760647414, ad_type = 1, length = 67,
>> contents = 0x14fd81a0 "0A0??\004\002\002"}
>> (gdb) p *enc_tkt_reply->authorization_data[1]
>> $3 = {magic = -1760647414, ad_type = 1, length = 67,
>> contents = 0x14fda180 "0A0??\004\002\002"}
>> (gdb) p *enc_tkt_reply->authorization_data[2]
>> $4 = {magic = -1760647414, ad_type = 1, length = 67,
>> contents = 0x14fd9820 "0A0??\004\002\002"}
>> (gdb) p *enc_tkt_reply->authorization_data[3]
>> Cannot access memory at address 0x31
>>
>> So, enc_tkt_reply->authorization_data[3] = 0x31, but it should either have
>> an allocated block of memory with an authdata item in it or a 0x00 to
>> terminate the authorization_data array.
>>
>> The request authorization data looks like this:
>>
>> (gdb) p *enc_tkt_request->authorization_data[0]
>> $6 = {magic = -1760647414, ad_type = 1, length = 67,
>> contents = 0x14fd7630 "0A0??\004\002\002"}
>> (gdb) p *enc_tkt_request->authorization_data[1]
>> $7 = {magic = -1760647414, ad_type = 1, length = 67,
>> contents = 0x14fd9770 "0A0??\004\002\002"}
>> (gdb) p *enc_tkt_request->authorization_data[2]
>> $8 = {magic = -1760647414, ad_type = 1, length = 54,
>> contents = 0x14fd97e0 "0402?\004\002\002\002"}
>> (gdb) p *enc_tkt_request->authorization_data[3]
>> Cannot access memory at address 0x0
>>
>> And it looks like we have the following for request authdata:
>>
>> authorization_data[0]: ad_type = KRB5_AUTHDATA_IF_RELEVANT
>> inside the item, we have ad_type = 141 (which doesn't have a
>> KRB5_AUTHDATA_ #define, but from google, it looks like it's
>> KERB_AUTH_DATA_TOKEN_RESTRICTIONS).
>> Inside that, we have what I assume is the KERB-AD-RESTRICTION-ENTRY.
>>
>> Here are the raw contents:
>> (gdb) p *enc_tkt_request->authorization_data[0]
>> $12 = {magic = -1760647414, ad_type = 1, length = 67,
>> contents = 0x14fd7630 "0A0??\004\002\002"}
>> (gdb) x/67xb enc_tkt_request->authorization_data[0].contents
>> 0x14fd7630: 0x30 0x41 0x30 0x3f 0xa0 0x04 0x02 0x02
>> 0x14fd7638: 0x00 0x8d 0xa1 0x37 0x04 0x35 0x30 0x33
>> 0x14fd7640: 0x30 0x31 0xa0 0x03 0x02 0x01 0x00 0xa1
>> 0x14fd7648: 0x2a 0x04 0x28 0x01 0x00 0x00 0x00 0x00
>> 0x14fd7650: 0x20 0x00 0x00 0x12 0x45 0x9d 0x93 0x95
>> 0x14fd7658: 0x0f 0x23 0x2b 0xca 0xf9 0x8a 0x86 0xf5
>> 0x14fd7660: 0x96 0x06 0xeb 0x79 0x5b 0x3e 0x3c 0x84
>> 0x14fd7668: 0x87 0xda 0x3c 0x04 0x0b 0xee 0x9d 0xd5
>> 0x14fd7670: 0xf0 0x0d 0x8c
>>
>> authorization_data[1] is identical to authorization_data[0]
>>
>> authorization_data[2]: ad_type = KRB5_AUTHDATA_IF_RELEVANT
>> inside that item, we have ad_type = 512 (KRB5_AUTHDATA_SIGNTICKET)
>> and inside that, we have the following, which doesn't quite look
>> like nested authdata, but what do I know?. Here are the raw contents:
>>
>> (gdb) p *enc_tkt_request->authorization_data[2]
>> $14 = {magic = -1760647414, ad_type = 1, length = 54,
>> contents = 0x14fd97e0 "0402?\004\002\002\002"}
>> (gdb) x/54xb enc_tkt_request->authorization_data[2].contents
>> 0x14fd97e0: 0x30 0x34 0x30 0x32 0xa0 0x04 0x02 0x02
>> 0x14fd97e8: 0x02 0x00 0xa1 0x2a 0x04 0x28 0x30 0x26
>> 0x14fd97f0: 0xa0 0x03 0x02 0x01 0x10 0xa1 0x1f 0x30
>> 0x14fd97f8: 0x1d 0xa0 0x03 0x02 0x01 0x0c 0xa1 0x16
>> 0x14fd9800: 0x04 0x14 0x4f 0xdd 0x1c 0x10 0x9e 0x9c
>> 0x14fd9808: 0xb5 0x82 0xec 0xf4 0x81 0x28 0xf2 0xa0
>> 0x14fd9810: 0xa0 0x78 0x70 0x48 0x8e 0xe9
>>
>> With all the plug-ins that process the authdata and build the reply
>> authdata (handle_tgt_authdata and handle_kdb_authdata are called before
>> handle_signedpath_athdata, I think), I can't quite tell which one might
>> be corrupting enc_tkt_reply->authorization_data[3].
>>
>> I was wondering if anyone has had similar problems with Krb5 1.8 processing
>> Windows authdata or any suggestions on isolating the cause of the problem?
>> Not being a Windows person, I'm already way beyond my knowledge of how
>> Windows, AD, and Kerberos interact.
>>
>> Thanks for any help you can provide.
>> --Mike Roszkowski
>> University of Wisconsin-Madison
>> ________________________________________________
>> Kerberos mailing list Kerberos at mit.edu
>> https://mailman.mit.edu/mailman/listinfo/kerberos
>
>
>
More information about the Kerberos
mailing list