design choices for a loadable module interface

[Nicolas Williams]

On Tue, Jun 29, 2010 at 04:20:05PM -0700, Russ Allbery wrote:
> Nicolas Williams <Nicolas.Williams at oracle.com> writes:
> > On Tue, Jun 29, 2010 at 03:19:25PM -0700, Russ Allbery wrote:
> 
> > That.  V-tables need to be written.  And they make it harder to read the
> > source, particularly since you're almost guaranteed to have different
> > function names in every plugin, so now you have to search for
> > assignments in cscope.  And they're ugly (yes, an aesthetic argument).
> 
> Hm, okay.  I personally just don't agree with this, I guess.  I've been
> writing plugins like that for a while, such as for Apache, and I don't see
> any problems with writing vtables or with them making the source hard to
> read.

Perhaps it's all just a matter of aesthetics then.

> >> I believe the chances of successfully holding to such a decision are
> >> zero.  There will be backwards-incompatible changes to function
> >> signatures and semantics.  The framework should plan for them from the
> >> start.
> 
> > Zero because... we can't be expected to not make such a mistake?  Or
> > because you don't want that constraint?
> 
> I don't think there's any chance that we'll anticipate all the
> requirements of a plugin ABI in advance and not have to ever change it in
> ways that aren't backward-compatible.  It's not just a matter of mistakes,

But why isn't it enough to deprecate the old and add new stuff?  I've
yet to find a situation where that wouldn't be good enough.

> although of course those are also not unlikely.  It's a matter of software
> evolving over time, which adds new requirements or enforces changes in the
> data that's passed through ABIs, and the plugin ABIs are brand new and
> don't have the practical experience that the Kerberos ABIs have.

It really, really helps to not have any structs in the ABI.  Things like
principal names, credentials, authentication contexts, etcetera -- all
of these should be opaque.

Kerberos is a bad example in that we're just not going to change those
structs (MIT considers the ABI to be stable).

> > Note that you could make the same kind of mistakes with a versioned
> > v-table scheme too if you're not careful, especially if you allow
> > plugins to support more than one version at a time.
> 
> Yes.  Either way, one definitely has to be careful.

So neither has an advantage in that regard :)

> > For user-land I much prefer (1); formative experience there includes,
> > among others, libpam and libgss (see below) (on Solaris).
> 
> PAM is, IMO, a bad example here, given that development of the PAM ABI is
> completely moribund despite a wealth of fundamental problems with the
> existing ABI that should really be fixed.

Sortof.  On the one hand what you say is true.  On the other hand many
of us have spent count-less hours designing PAM extensions -- if only we
could get that work funded...

> > On Solaris libgss uses a non-version v-table for old stuff, dlsym() for
> > new stuff.  Every time we added functions we had to patch libgss and all
> > plugins together.  What a pain.  Granted, with a versioned v-table we'd
> > not have to, but now we'd have N versions of the same crappy structure
> > lying around, with libgss having to know all N -- that's a lot of
> > unnecessary duplication and unnecessary code, plus it's ugly.  So I just
> > made it so new additions are dlsym()ed and that's that.
> 
> If you believe the most likely change is to add a new function that's
> unrelated to all existing functions, and if you think that's likely to
> happen frequently, I can definitely see why you would prefer a
> function-driven interface.  You otherwise end up with lots of versions of
> vtable symbols since you're reving that version each time you introduce a
> new function.  I don't believe that's likely to be the case for Kerberos
> plugins, though.  I think the most likely changes will be changes to the
> signature or behavior of an existing interface, rather than the addition
> of a new interface unrelated to existing interfaces in that module.

We've certainly added new versions of old functions.  The old remain or
are deprecated, the new provide more or alternative functionality.

If you look around you'll find that long-lived APIs are littered with
*2, *ext, *_ext, ... variants, all different in relatively minor ways.
I used to think that's ugly, but it's not: the callers change more
slowly than the API, thus the need for backwards-compatibility.  The
same applies to plugin interfaces.

In fact, the simplest solution by far is to say that versioning is out
of scope: it's the packaging system's problem.  (In fact, that's how
libgss and its plugins used to be versioned in Solaris.)  It helps if
the interface is private, but it's not necessary, provided that the
packaging system is sufficiently advanced.

> Of course, to some extent one can transform one problem into the other.

I find (1) and (2) equivalent: they both will get the job done, and they
both had similar issues.  (2) is ugly in my book.

I've said all there is for me to say on this subject.  You can have the
last word, and whatever MIT chooses I'll deal.

Nico
--