problems with making calls to gssapi32.dll on Windows XP

[Douglas E. Engert]

David Bienvenu wrote:

> Hi,
> 
> My name is David Bienvenu, and I work on mail&news for the Mozilla 
> Foundation. I'm trying to make calls to gssapi32.dll from Mozilla code, 
> to add MIT Kerberos support to Mozilla Mail and Thunderbird (My 
> understanding is that using SSPI on Windows won't work with MIT's KDC, 
> and I want to support MIT's KDC since I think that's widely used). I 
> took Mozilla's existing negotiateuth code ( 
> http://bonsai.mozilla.org/rview.cgi?cvsroot=/cvsroot&dir=mozilla/extensions/negotiateauth&module=default&rev= 
> ) and flipped some makefile switches so that we'd use GSSAPI on Windows, 
> like we do successfully for linux and mac. I had to add gssapi32 to the 
> list of libraries we look for.  After that, our gssapi code found the 
> right entry points and was able to call them successfully. But, the out 
> parameters to those calls, e.g., gss_indicate_mechs_ptr, initially look 
> OK, but as I execute code after the call, the values become corrupted, 
> and we crash. For example, calling gss_release_oid_set_ptr crashes 
> because the gss_OID_set mech_set either gets nulled out, or corrupted in 
> weird ways. It's bizarre to step through the vc 6.0 debugger and 
> eventually see the local variables change value even though the 
> disassembled code doesn't change the local vars. I'm wondering if you 
> have any idea what could be going on, or point me to someone who might 
> know. I would suspect some sort of calling convention problem, but we're 
> using the standard cdecl calling convention.
>

Wrong calling convention. Its __stdcall on most. See the gssapi.h
that comes with the Kerberos for Windows from MIT.

> The gssapi32.dll in windows/system32 has a version of 1.2.3.0, created 
> 3/20/2002. I believe the Kerberos version is 2.1.2.0, which is what the 
> institution I'm trying to help uses. I'm using the vc 6.0 dev tools, and 
> we compile with fairly vanilla flags, /Zi, basically. This is a debug 
> build of Mozilla.
> 

That is pretty old version of Kerberos, and does not support TCP, so
will have problems with Windows. But the gssapi calling conventions have
not changed.




> thx for any pointers you can give,
> 
> - David Bienvenu
> _______________________________________________
> krbdev mailing list             krbdev at mit.edu
> https://mailman.mit.edu/mailman/listinfo/krbdev
> 
> 
> 

-- 

  Douglas E. Engert  <DEEngert at anl.gov>
  Argonne National Laboratory
  9700 South Cass Avenue
  Argonne, Illinois  60439
  (630) 252-5444