[krbdev.mit.edu #1263] Interoperability compatibility between Heimdal and MIT krb5

[Ezra Peisach via RT]

Issue:  MIT is using integers for sequence numbers and Heimdal is using
unsigned integers.

As such - MIT is encoding sequence numbers as signed - while Heimdal is
trying to decode them.
Heimdal does not handle a first octet being signed and interpreting as a
potentially signed value.

What this will mean - until MIT starts sending out unsigned encodings is
that there is a potential
for mis-interpretation of seqeuence numbers. 

I do not know what MS is doing - or anyone else - but if they are basing
code on the original specs
then we must assume that we will be getting sequence numbers as signed
integers and be prepared to 
handle them.

MIT is currently encoding a sequence number 0xFF800000 as "02 03 80 00
00" which is valid 
for a signed value. Heimdal is decoding this as "0x800000" (Side note:
Heimdal's unsigned integer 
encoder is violating X.690 with this - emitting 02 04 FF 80 00 00 -
which violates the rule
that the first octet and the fhigh bit of the second octet cannot all be
one - I believe 02 05 00 FF 80 00 00 is the proper encoding for such a
value in an unsigned sense)

(As a separate note - MIT's unsigned integer decoder will reject "02 03
80 00 00"  - because it doe not handle signed values).

I am not sure what the long term fix for this signed vs unsigned issue
is.  I guess it depends on the
code base out there - but being liberal in what one receives and strict
in what one sends - we will
probably need to be able to accept signed encodings for unsigned
elements for some time.

--Ezra