RC4 Weak Key checks

[Jeffrey Altman]

In k5_arcfour_init() src/lib/crypto/enc_provider/rc4.c MIT's
implementation of RC4 has a check for two families of weak keys.
Those beginning with either of the following three octet sequences:

  arcfour_weakkey1[] = {0x00, 0x00, 0xfd};
  arcfour_weakkey2[] = {0x03, 0xfd, 0xfc};

If a key in either of these families is detected in k5_arcfour_init(),
the error KRB5DES_WEAK_KEY is returned to the caller.

In my reading of RFC 3961 and RFC 4757 I do not come across any
indication that a weak key test should be applied when RC4 is used with
Kerberos or GSS.  When used with GSS the weak key test is especially
problematic.  For example, each call to gss_wrap() generates a new per
message RC4 key as each message is treated by RFC 4757 as a new RC4
keystream.  The RC4 key is generated using the sequence number as input
using the following call sequence:

  gss_wrap -> gss_seal -> k5glue_seal -> krb5_gss_seal -> kg_seal ->
  make_seal_token_v1 -> kg_make_seq_num -> kg_arcfour_docrypt ->
  k5_arcfour_docrypt -> k5_arcfour_init

An application calling gss_wrap() experiences a random behavior.  When a
KRB5DES_WEAK_KEY error is returned from k5_arcfour_init(), it is never
handled and the gss_wrap() call fails.  This can happen relatively
quickly or can require hundreds of thousands of messages.  Regardless,
if the GSS context is used to send enough messages, the error will be
thrown.

In examining other implementations of RC4 I can find no weak key check.
 I am questioning whether MIT's implementation should have one and if so
whether it should be used in conjunction with GSS.

If the weak key check is maintained, it implies that in any given GSS
context certain sequence numbers cannot be used.  Since there is no
standard for how these sequence numbers should be skipped, the inclusion
of the weak key check is an interoperability problem with existing
deployed GSS implementations.

I would appreciate the feedback of the members of this list.

Jeffrey Altman

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: OpenPGP digital signature
Url : http://mailman.mit.edu/pipermail/krbdev/attachments/20110324/da99d22a/attachment.bin