krb5_rd_req return codes for ticket decryption failures

[Greg Hudson]

I am working on some changes to produce better error messages from
krb5_rd_req, and while I'm there I want to take a look at its return
codes.

As of 1.7, we return KRB5KRB_AP_WRONG_PRINC for most ticket decryption
failures.  Despite the name, this code does not correspond to an RFC
4120 error number, so the krb5 mech's gss_accept_sec_context will
translate it into KRB_ERR_GENERIC (60) in the KRB-ERROR message.  It
seems like we would be better off returning actual AP-REQ protocol
errors (especially because of [1]).  My tentative plan is:

* KRB5KRB_AP_ERR_NOKEY: The keytab is missing or unreadable, or contains
  no usable entries (i.e. none of the keytab principals match the
  application server specification), so we can't decrypt any tickets.

* KRB5KRB_AP_ERR_NOT_US: The keytab does not contain any usable entries
  for ticket server principal (and we tried all of the usable entries in
  case the ticket used an alias, but none of them worked).

* KRB5KRB_AP_ERR_BADKEYVER: The keytab contains a usable entry for the
  ticket server principal, but it doesn't match the ticket kvno and
  enctype (and we tried all of the usable entries anyway but none of
  them worked).

* KRB5KRB_AP_ERR_INTEGRITY: The keytab contains a key matching the
  ticket server principal, kvno, and enctype, but it didn't work (nor
  did any of the other usable entries).

If the ticket used an alias for a keytab entry we do have, but uses the
wrong kvno, we can't really tell that it was a kvno mismatch, so we
return a principal mismatch.  This is a fundamental limitation of RFC
6806 server aliases.

We can distinguish between different sub-cases using the extended error
message, but that won't be visible to the client.  For the purpose of
this thread, I'm only discussing the return code.

>From [2] I'm aware that we have some old GSSRPC code in the tree which
reacts to KRB5KRB_AP_WRONG_PRINC; it can be changed to react to
KRB5KRB_AP_ERR_NOT_US.

Does this plan seem reasonable?

[1] http://k5wiki.kerberos.org/wiki/Projects/Graceful_recovery_after_destructive_service_rekey
[2] http://mailman.mit.edu/pipermail/krbdev/2008-December/007154.html