GSSAPI interoperability problem between Java 1.5 & MIT Kerberos

Ken Hornstein kenh at
Mon Jan 9 15:06:43 EST 2006

One of our developers has discovered what I believe is a significant
GSS interoperability problem between the latest version of Java and MIT
Kerberos.  Specifically ...

Our developer was using a Java application with Java 1.4 to talk to an
LDAP server using Cyrus-SASL and some version of RedHad's Kerberos
library (I mention this so it's clear it wasn't our wacky patched Kerberos
library that was in the mix).  This program worked fine.

When he upgraded to Java 5 (which I guess is also known as "Java 1.5"),
it failed ... specifically, Cyrus-SASL on the application server would
return, "A token is invalid".

After some packet traces and judicious use of the debugger later, we
discovered the problem was that Java 1.4 used as it's integrity algorithm
(SGN_ALG) DES-MAC-MD5 (0x0000).  However, Java 1.5 now uses DES-MAC
(0x0200).  Part of me thinks this was a reaction to one of the announced
MD5 vulnerabilities, but I have no idea; I am just guessing.

On the application server, this algorithm fails in k5unseal.c, around line

    switch (signalg) {
    case SGN_ALG_DES_MAC_MD5:
    case SGN_ALG_MD2_5:
    case SGN_ALG_HMAC_MD5:
        cksum_len = 8;
        if (toktype != KG_TOK_SEAL_MSG)
          sign_usage = 15;
    case SGN_ALG_3:
        cksum_len = 16;
        cksum_len = 20;
        *minor_status = 0;
        return GSS_S_DEFECTIVE_TOKEN;

Obviously, it falls out through this switch statement and ends up returning

RFC 1964 says regarding DES-MAC: "Support for this algorithm may not be
present in all implementations."  RFC 4121 has changed things around and
is silent on the issue.  I believe that Java is wrong in using DES-MAC
for integrity, but what do other people think?  More importantly, is there
a way to fix it?


More information about the krbdev mailing list