Initial comments request: AEAD Encryption API

[Nicolas Williams]

On Wed, Nov 05, 2008 at 06:11:11PM -0500, Ken Raeburn wrote:
> On Nov 5, 2008, at 11:18, Sam Hartman wrote:
> > Folks, I'm helping Luke Howard who is working on getting a number of
> > Microsoft extensions integrated into MIT Kerberos.  I'm putting
> > together documentation sufficient for our review of the work as well
> > as reviewing his code.
> >
> > I'd like to get some early comments on
> > http://k5wiki.kerberos.org/wiki/Projects/AEAD_encryption_API .
> 
> A spec for the crypto stuff would help.  And API docs.

+1

> > The API exposes padding separate from trailers. However RFC 3961 has  
> > neither concept. Is this the right balance?
> 
> Once you start doing scatter-gather and in-place encryption, it's hard  
> to keep things abstract.  RFC 3961 imposes a few arbitrary-looking  
> constraints, but having easily separated padding and trailers, and an  
> identifiable integrity-check tag, is not among them.

A revision certainly seems likely to be needed.

First, to specify AEAD by generic composition for the existing enctypes.

Second, to specify new enctypes that use AEAD modes of AES and avoid
generic composition.

There may not be enough interest in the latter.

> > With the existing crypto systems, the order between data chunks and  
> > sign_only chunks matters. That is, (data|sign_only|data) will  
> > produce a different checksum than (sign_only|data|data). Do we want  
> > callers to depend on this, or will doing so create a problem if we  
> > have a crypto system based on RFc 5116? I suspect that we want to  
> > say that a crypto system may treat the sign_only chunks as one  
> > ordered sequence and the data chunks as another ordered sequence but  
> > not require that the relative ordering between data chunks and  
> > sign_only chunks affect the cipher text.

I'd have to re-read RFC5116.  Certainly being compliant with it would be
nice.

> It already looks like the protection is kind of weak for two  
> consecutive chunks of the same type; an attacker could push data  
> around and adjust some lengths, and ciphertext and plaintext wind up  
> moving from one block to another, with no change to the checksum.  If  
> the chunking is purely for convenience and not intended to be  
> protected or to carry meaning, then yes, I think perhaps we could  
> define something based on RFC 5116.

Chunks should be logically concatenated.

> > The API design assumes fixed-length headers, padding and trailers.
> 
> It also looks like for decryption you have to know how to separate the  
> different components (e.g., ciphertext from trailer) before decrypting.

I'd rather not have as constrained an API as the SSPI was.  Let there be
functions that the caller can call to determine the lengths of the
headers, padding and trailers, but do let's assume that for any one
context (enctype) they are, in fact, fixed.

> > Implementation notes
> > While the API supports in-place encryption the implementation does  
> > not significantly reduce the number of copies.
> 
> We should decide if in-place encryption is important to us.  Forcing  
> the application to explicitly copy an input it wants to use multiple  
> times is annoying, but in-place encryption can help memory-constrained  
> environments.

In-place crypto helps a lot.  Please leave it in.  (Lots of HW crypto
accelerators support in-place crypto, but I suspect that the really
important thing is L2 cache pressure.)

> I can imagine some specific uses for separate inputs and outputs with  
> scatter-gather.  For example, Tom and I have discussed a possible ASN. 
> 1 encoder scheme that tracks the various bits of data to be put  
> together (tags, encoded numbers, etc), but avoids actually copying the  
> application data (strings, buffers) until assembling the final result  
> (instead of our current annoying copy-backwards-and-reverse-at-the-end  
> approach); for a message part to be encrypted, with scatter-gather  
> input and a big buffer output, we could avoid the copying altogether  
> except as it happens in the encryption process.  Requiring it to be in- 
> place means copying the application data to temporary storage before  
> encrypting, like we do now.

Providing support for in-place crypto != requiring use of in-lace crypto
APIs.

Crypto HW might have scatter/gather chunk alignment requirements, but if
you have mostly large chunks then you can internally create a larger
chain of chunks and copy only small amounts of data and still take
advantage of in-place crypto.

> > The implementation is parallel to the existing implementation. In  
> > particular, the existing APIs are not implemented in terms of the  
> > new APIs. New code is required (with significant copying) from the  
> > generic API layer down to the enc_provider layer, although of course  
> > raw crypto primitives are used.

If that's for performance reasons, then that's quite understandable.

> > Currently sanity checking buffers--for example making sure there is  
> > a trailer chunk--is left to the enc_provider implementations.
> 
> Currently, there's no AEAD for DES, so we can't just build generic  
> encryption on top of generic AEAD.  And having to separate ciphertext  
> and trailers means building generic decryption on top of generic AEAD  
> would be kind of ugly even if we had the DES support, unless we  
> implement a new chunk type for "everything thrown together", or a way  
> to pull out the lengths of the different parts.

I think you could define AEAD for DES by generic composition and then
have the old routines use the new AEAD ones, but without using the "AD"
part.  That'd work, if you're careful.  I think that's what Sam meant.
But I'm not sure that it's worth doing just for reducing code
duplication since it will add overhead to the most common cases.

Nico
--