krb5-1.8 fails to verify MS PAC Checksum when AES 256 is used causing sshd to fail

[Douglas E. Engert]

On 7/1/2010 4:35 PM, Luke Howard wrote:
>> With  msDS-SupportedEncryptionTypes = 16 (AES256) The first verify fails
>> as expected, and the keytab is searched, and each key is tried. But
>> the RC4 key (23) gets a KRB5KRB_AP_ERR_BAD_INTEGRITY as the compare
>> of the computed and supplied checksums don't match.
>
> Perhaps they're rc4-hmac with the AES key. (This really wouldn't surprise me. Ironically it might make the code path simpler.)
>

I was thinking along the same lines last night. I tried you second patch, and that did not work
getting these messages:
   GSS-API error accepting context: Unspecified GSS failure.  Minor code may provide more information
   GSS-API error accepting context: Key size is incompatible with encryption type


But here is a patch that does work (only minimal testing was done):

--- lib/crypto/krb/,cksumtypes.c        Thu Dec 10 11:10:10 2009
+++ lib/crypto/krb/cksumtypes.c Fri Jul  2 09:16:32 2010
@@ -82,8 +82,8 @@

      { CKSUMTYPE_HMAC_MD5_ARCFOUR,
        "hmac-md5-rc4", { "hmac-md5-enc", "hmac-md5-earcfour" },
-      "Microsoft HMAC MD5 (RC4 key)",
-      & krb5int_enc_arcfour, & krb5int_hash_md5,
+      "Microsoft HMAC MD5 (RC4 or AES keys)",
+      NULL, &krb5int_hash_md5,
        krb5int_hmacmd5_checksum, NULL,
        16, 16, 0 },


This removes the enc routine as it is not needed. The verify_key was
was producing the bad encryption type error before, as it was checking
id an enc routine was required.

The CKSUMTYPE_HMAC_MD5_ARCFOUR = -138 may be misnamed, as it in
not depeendent on RC4. As RFC 4757  calls it:

#define KERB_CHECKSUM_HMAC_MD5 (-138)

The CKSUMTYPE_MD5_HMAC_ARCFOUR = -137 may also be misnamed, and the
same patch might be needed with it as well. I don't know where is is
used, but since HMAC can use any key <= 64 bytes, it may not need the
& krb5int_enc_arcfour.



> -- Luke
>

-- 

  Douglas E. Engert  <DEEngert at anl.gov>
  Argonne National Laboratory
  9700 South Cass Avenue
  Argonne, Illinois  60439
  (630) 252-5444