GSS unwrap fails using RC4 session key instead of subkey

[Michael B Allen]

On Thu, May 8, 2025 at 12:10 AM Michael B Allen <ioplex at gmail.com> wrote:

> So the session key used starts with C952.
>

I meant to say subkey not session key.

For completeness I ran my initiator against the Windows Server SSPI
acceptor, and got the following:

InitStepState: Authenticator {
  cname: TsspiUserAes256 at MEGA.CORP
  cksum: Checksum8003 {
    channelBindings: 00000000000000000000000000000000
    gssflags: 0x0000403E
        GSS_C_DELEG_FLAG
      Y GSS_C_MUTUAL_FLAG
      Y GSS_C_REPLAY_FLAG
      Y GSS_C_SEQUENCE_FLAG
      Y GSS_C_CONF_FLAG
      Y GSS_C_INTEG_FLAG
        GSS_C_DCE_STYLE
        GSS_C_IDENTIFY_FLAG
      Y GSS_C_EXTENDED_ERROR_FLAG
  }
  cusec: 588294
  ctime: 20250508174734Z
  subkey: (23)412213...
  seq-number: 656590050
}
InitStepState: GssContextToken {
  mech: KRB5 (1.2.840.113554.1.2.2)
  Krb5InnerContextToken {
    tokId: 0x0001
    ApReq {
      ap-options: 0x20000000
      ticket: Ticket {
        sname: HOST/TsspiCompRc4.mega.corp at MEGA.CORP
        enc-part: (23)6D9EDF...
      }
      authenticator: (23)3B0B5F...
    }
  }
}
InitStepState: GssContextToken {
  mech: KRB5 (1.2.840.113554.1.2.2)
  Krb5InnerContextToken {
    tokId: 0x0002
    ApRep {
      enc-part: (23)8A32FD...
    }
  }
}
InitStepState: EncAPRepPart {
  ctime: 20250508174734Z
  cusec: 588294
  subkey: (23)412213...
  seq-number: 2110239284
}
Rc4 wrap: key:
00000: 41 22 13 30 34 0D D6 44 39 7E 27 E5 91 31 30 1D  |A".04.ÖD9~'å.10.

As you can see, the SSPI acceptor simply uses the same key for the
Authenticator subkey and AP-REP subkey.
Not sure how the SSPI knows to do this.
Maybe it's just hardcoded behavior of RC4.

So when MITK initiates to an SSPI RC4 service, it uses the Acceptor subkey
whereas the SSPI initiator will use the AP-REP subkey but it doesn't matter
because the keys are the same.

The bottom line is that if anyone writes an acceptor from scratch like me,
and they want to support RC4, the acceptor will need to either "Negotiated
enctype" to a different enctype so that the MITK initiator uses the AP-REP
subkey (the MITK way), or just return the same RC4 subkey for the
Authenticator and AP-REP subkeys (the SSPI way) so that does not matter if
the MITK initiator uses the Authenticator subkey.

Mike

-- 
Michael B Allen
Java AD DS Integration
https://www.ioplex.com/ <http://www.ioplex.com/>