MIT Kerberos 1.12.4 kinit fails in AIX

[Vipul Mehta]

Hi,

Following are my setup details :
=> AIX version 6
=> MIT Kerberos version 1.12.4
=> Windows Server 2008 KDC.

Our kerberos authentication API dynamically loads MIT Kerberos and calls
its APIs via function pointer. It works fine in Linux.64 and Windows.

We have done two file changes for building MIT Kerberos in AIX:
    1) MitKerberos/1.12.4/src/include/k5-platform.h
    350,351c350,354
    <
    < #if !defined(SHARED) && !defined(_WIN32)
    ---
    > /*
    >  * Making a fix for AIX: as we don't want finish to be static
function in
    >  * case of AIX platform. Build fails otherwise.
    >  */
    > #if !defined(SHARED) && !defined(_WIN32) && !defined(_AIX)


    2) MitKerberos/1.12.4/src/plugins/kdb/db2/Makefile.in
    3a4
    > UNAME=$(shell uname)
    27a29
    > LDBLIBDIR=./libdb2
    34c36,41
    < SHLIB_EXPLIBS= $(GSSRPC_LIBS) -lkrb5 -lcom_err -lk5crypto
$(KDB5_DB_LIB) $(KADMSRV_LIBS) $(SUPPORT_LIB) $(LIBS) @DB_EXTRA_LIBS@
    ---
    >
    > ifeq ($(UNAME), AIX)
    >         SHLIB_EXPLIBS= $(GSSRPC_LIBS) -lkrb5 -lcom_err -lk5crypto
$(KDB5_DB_LIB) $(KADMSRV_LIBS) $(SUPPORT_LIB) $(LIBS) -L$(LDBLIBDIR)
@DB_EXTRA_LIBS@
    > else
    >         SHLIB_EXPLIBS= $(GSSRPC_LIBS) -lkrb5 -lcom_err -lk5crypto
$(KDB5_DB_LIB) $(KADMSRV_LIBS) $(SUPPORT_LIB) $(LIBS) @DB_EXTRA_LIBS@
    > endif

Issue 1 : kinit fails with illegal instruction
=> Our kerberos authentication API which uses MIT Kerberos was failing
while acquiring keytab credentials. I found that kinit executable generated
on building MIT Kerberos itself fails in any case (username-password,
keytab, simple execution without arguments). On debugging i found some
memory corruption in dnsglue.c->krb5int_dns_init(). After the malloc call
in this function, the arguments values got corrupted.
Reverting following commit  fixed this issue (how ?):
https://github.com/krb5/krb5/commit/a505552a8d9d1c0e228236ae7f5f0422175f3b62?diff=split


Issue 2 (after fixing issue 1) : On running kinit, assertion error occurs @
k5-thread.h->k5_mutex_lock()
The assert subroutine failed: r == 0, file ../../include/k5-thread.h, line
384

Stacktrace  :
Breakpoint 1, dereference (context=0x0, data=0x0) at cc_file.c:1478
1478        k5_cc_mutex_lock(context, &krb5int_cc_file_mutex);
(gdb) bt
#0  dereference (context=0x0, data=0x0) at cc_file.c:1478
#1  0x0900000008a3ecdc in krb5_fcc_close (context=0x0, id=0x0) at
cc_file.c:1518
#2  0x0900000008a024ec in krb5_cc_close (context=0xbadc0ffee0ddf00d,
cache=0xbadc0ffee0ddf00d) at ccfns.c:76
#3  0x00000001000014e0 in k5_end (k5=0x0) at kinit.c:668
#4  0x00000001000009cc in main (argc=0, argv=0x0) at kinit.c:931
(gdb) s
The assert subroutine failed: r == 0, file ../../include/k5-thread.h, line
384

Program received signal SIGABRT, Aborted.
0x0900000000bb37d0 in pthread_kill () from
/usr/lib/libpthreads.a(shr_xpg5_64.o)

I checked and the return code from k5-thread.h->k5_os_mutex_lock() is 22
i.e. invalid mutex.

Anyone got any idea what's going wrong here ?

Regards,
Vipul