Future ASN.1 support

[ghudson@MIT.EDU]

We have several problems with the current state of ASN.1 encoding and
decoding in MIT krb5:

* Plugin modules and applications cannot make use of the libkrb5 code
to encode and decode types built on top of Kerberos types.  In the
past we have incorporated support into the core code when a module
needs to do ASN.1 encoding, which kind of defeats the point of having
a pluggable interface.

* Translating from an ASN.1 module to the data-driven encoder
structures is painstaking and error-prone.

* Decoders are still open-coded using macros.  Translating from an
ASN.1 module to decoding functions is also painstaking and
error-prone.

* Our table-driven encoding logic and decoding macros do not hew very
closely to the ASN.1 data model; instead, they are designed around the
idioms of RFC 4120 (and to some extent to the krb5 type hierarchy).
As we incorporate types from different ASN.1 subcultures, we wind up
with hacky workarounds.  For instance, some PKINIT types are still
open-coded, partly because they use some implicit tags.

* Writing good tests for ASN.1 encoders and decoders is tedious.  The
risk of interoperability issues is extremely high if we don't generate
test cases using a separate (and good) ASN.1 implementation.

I want to address this problem for 1.11.  Here are the broad options
I'm considering:

1. asn1c + type conversion: we could use asn1c to generate structures
and encoders for ASN.1 types, and then wrap the encoders and decoders
with converters for our types.  (Using asn1c's types directly would be
a huge migration issue and would also be pretty cumbersome.)

2. Heimdal asn1_gen + hacking: we could incorporate the Heimdal
libasn1 and modify it to generate code for our types and handle our
edge cases.

3. New ASN.1 compiler + modified existing code: in this plan we alter
our ASN.1 infrastructure to (1) be less tailored to RFC 4120 idioms,
(2) support data-driven decoding as well as encoding, and (3) throw
out the CPP macros for generating data structures.  Then we write a
new ASN.1 compiler in Python which generates the data structures based
on an ASN.1 module and some additional input describing our data
structures.

For all of these options, we also need to expose some of the ASN.1
machinery so that plugins can compile their own ASN.1 modules which
import existing types.

Currently I'm exploring the third option, but I'm interested in
opinions on which option is best or whether I should be looking at
other options.