make check fails on AIX 5.3

[Gavin Sherry]

Hi all,

I am able to build Kerberos 1.7 on AIX 5.3 (both 32 and 64 bit). However,
neither build passes make check.

configure is only being passed the following: --without-tcl. GCC is version
4.2.4.

The checks which fail are:

t_locate_kdc ATHENA.MIT.EDU
walktree-tests
testdb -r FOO.TEST.REALM -P footes create -s

These all fail in the same way: a assertion fails and core is dumped.

Assertion failed: __EX, file  threads.c, line 350

The back trace always looks something like the follow (here, specifically
for t_locate_kdc):

#0  0xd01247d4 in pthread_kill () from /usr/lib/libpthreads.a(shr_xpg5.o)
#1  0xd0124248 in _p_raise () from /usr/lib/libpthreads.a(shr_xpg5.o)
#2  0xd0356bdc in raise () from /usr/lib/libc.a(shr.o)
#3  0xd03b4efc in abort () from /usr/lib/libc.a(shr.o)
#4  0xd03bfe70 in __assert_c99 () from /usr/lib/libc.a(shr.o)
#5  0xd24af644 in krb5int_key_register (keynum=K5_KEY_COM_ERR,
destructor=@0xf03cfbb4: 0xcf94) at threads.c:350
#6  0xd2201d48 in com_err_initialize () at error_message.c:58
#7  0xd2201c80 in com_err_initialize__aux () at error_message.c:42
#8  0xd0115fb8 in pthread_once () from /usr/lib/libpthreads.a(shr_xpg5.o)
#9  0xd22021a4 in error_message (code=-1429577725) at error_message.c:162
#10 0x10001e9c in krb5_locate_srv_conf_1 (context=0x20009988,
realm=0x2ff22a08, name=0x1000c490 "kdc", addrlist=0x2ff22940, get_masters=0,
socktype=0, udpport=88, sec_udpport=750,
    family=0) at locate_kdc.c:336
#11 0x10003064 in prof_locate_server (context=0x20009988, realm=0x2ff22a08,
addrlist=0x2ff22940, svc=locate_service_kdc, socktype=0, family=0) at
locate_kdc.c:743
#12 0x1000337c in krb5int_locate_server (context=0x20009988,
realm=0x2ff22a08, addrlist=0x20001550, svc=locate_service_kdc, socktype=0,
family=0) at locate_kdc.c:818
#13 0x10003584 in krb5_locate_kdc (context=0x20009988, realm=0x2ff22a08,
addrlist=0x20001550, get_masters=0, socktype=0, family=0) at
locate_kdc.c:862
#14 0x10003ae0 in main (argc=2, argv=0x2ff22ac8) at t_locate_kdc.c:123

The issue is that com_err_initialize() is being called more than once. The
first time, it is done via krb5int_initialize_library(), the second via the
bt above. This should be guarded by pthread_once(), but that's not working
(although, I've verified that in a trivial program pthread_once() is
behaving as expected).

Here's some more debugging info. Before the first call,
com_err_initialize__once is set to:

$2 = {once = {__on_word = {0, 0, 0, 0, 0, 0, 0, 2, 0 <repeats 20 times>}},
error = 0, did_run = 0, fn = @0xf08d61dc: 0xd21f3eb4
<com_err_initialize__aux>}

After the first call:

$4 = {once = {__on_word = {1, 0, 0, 0, 0, 0, 0, 9, 0 <repeats 20 times>}},
error = 0, did_run = 1, fn = @0xf08d61dc: 0xd21f3eb4
<com_err_initialize__aux>}

Before the second call:

$5 = {once = {__on_word = {2, 0, 0, 0, 0, 0, 270008407, 9, 1, 0 <repeats 19
times>}}, error = 0, did_run = 1, fn = @0xf08df2dc: 0xd21eefb4
<com_err_initialize__aux>}

Now, I set a watch point in the debugging to stop when this structure is
changed. GDB does not break between the first and second calls. This might
be a short coming of GDB, I'm not sure.

So, something screwy is happening with pthreads here. Other applications
which use pthreads appear to be working correctly so it's possible this is
something to do with KRB.

This is surely not enough information to go on. Please let me know what
other information I can provide so that we can track down the issue.

Thanks,
Gavin