KRB5_REALM_CANT_RESOLVE errors ?

[Christopher Burke]

raeburn at mit.edu (Ken Raeburn) wrote in news:tx1663hdlj3.fsf at mit.edu:

> Christopher Burke <craznar at hotmail.com> writes:
>> raeburn at mit.edu (Ken Raeburn) wrote in news:tx1eli5e885.fsf at mit.edu:
>> > You don't say just what routine is failing, but that error is only
>> > returned from the code for finding the KDC IP addresses.  You could
>> > add some debugging-printf code to locate_kdc.c at various points to
>> > see where it might be failing.  Also check the realm name that is
>> > supplied -- if that's not right, locate_kdc has no prayer of working.
>> 
>> Occurs in krb5_get_in_tkt_with_password, works find for 1000s and 1000s
>> of attempts.... but starts failing when there is a high volume of
>> requests. 
> 
> Can you reproduce the failure in a testing setup?

No - it only happens with high random load. My test script which fires 60 
or so threads of requests to my app doesn't crash it.

> Try adding the debugging code to locate_kdc, especially in any code
> paths where the CANT_RESOLVE error is actually returned.  Start
> logging all the output, and then start pushing it hard until it
> fails.

Debugging the live server kills it due to overwhelming log writes.

>> > You mention using one thread at a time -- is the application doing
>> > DNS stuff (explicitly or via gethostby*) in other threads at the same
>> > time as Kerberos authentication is being attempted?  That might
>> > confuse things.  In the 1.3 release I'm switching some things over to
>> > use getaddrinfo, which should be thread-safe on many platforms; that
>> > might help.
>> 
>> Its a multi-threaded app, with one funnel for kerberos auth - and a
>> mutex around all the kerberos stuff.
> 
> That's a start -- but as I said, the krb5 library uses some C library
> routines that may themselves not be thread-safe, and if you're using
> those routines also in other threads at the same time, that could
> cause problems.  (Gethostbyname and gethostbyaddr, for example.)

No - ALL kerberos stuff is housed within the mutex, one at a time through 
the funnel.

> Another variation you might test is a single-threaded test program
> that just runs thousands of authentication attempts in a loop without
> delay.  If that works okay, the multi-threaded aspect of your
> application code may be related to the problem.

The kerberos stuff is mutexed safely....

>> > If by "high load" you mean "many more threads running", there's a
>> > good chance it's related to the lack of thread safety not just in the
>> > MIT krb5 library, but in some of the C library functions it calls.
>> 
>> No by high load I mean very little time between one set of kerberos
>> calls (an entire authentication) and the next set. There is only one
>> auth going on at any given time in this app (which is iPlanet Directory
>> server by the way). 
> 
> Mere latency between calls shouldn't be important, unless it means
> some other service (local or remote named process?) isn't getting the
> cycles or memory it needs, or is implementing some sort of rate
> limiting to avoid denial of service attacks.

Don't know for sure yet... but that latency between calls seems to be the 
issue - but only on this 1 call at a time application. The multi threaded 
apps that fork out multiple kerberos 5 auths work fine.

-- 
---
/* Christopher Burke - Spam Mail to craznar at hotmail.com
|* www.craznar.com - 
\* Real mail to cburke(at)craznar(dot)com