[krbdev.mit.edu #8819] gss_set_allowable_enctypes() fails if any enctypes aren't recognized

[Greg Hudson via RT]

Mon Jul 15 22:29:17 2019: Request 8819 was acted upon.
 Transaction: Ticket created by ghudson at mit.edu
       Queue: krb5
     Subject: gss_set_allowable_enctypes() fails if any enctypes aren't recognized
       Owner: Nobody
  Requestors: ghudson at mit.edu
      Status: open
 Ticket <URL: https://krbdev.mit.edu/rt/Ticket/Display.html?id=8819 >


gss_set_allowable_enctypes() is typically used to force the session key
negotiated for a GSS context to fall within the supported set of another
implementation, such as the Linux kernel.

In the current implementation, any invalid enctype in the requested set causes
the call to fail. This can cause a problem if the external implementation
supports any enctypes we don't. In Debian bug #932000 this was observed because
the kernel still supports single-DES enctypes while we removed them. The
problem could also arise if the kernel supports enctypes too new for libkrb5,
such as if the kernel supports aes-sha2 enctypes but libkrb5 is from a release
before 1.15.

Of course the caller can do its own filtering with krb5_c_valid_enctype(), but
that's pushing work onto the caller for no reason. Instead,
gss_set_allowable_enctypes() should filter out invalid enctypes and only error
out if no enctypes remain.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=932000