design choices for a loadable module interface

[Russ Allbery]

Nicolas Williams <Nicolas.Williams at oracle.com> writes:
> On Tue, Jun 29, 2010 at 03:19:25PM -0700, Russ Allbery wrote:

>> Could you be more specific about exactly what functionality you see the
>> vtable approach as duplicating?  I assume you mean something more than
>> just writing the struct definition of the vtable, since that's trivial.

> That.  V-tables need to be written.  And they make it harder to read the
> source, particularly since you're almost guaranteed to have different
> function names in every plugin, so now you have to search for
> assignments in cscope.  And they're ugly (yes, an aesthetic argument).

Hm, okay.  I personally just don't agree with this, I guess.  I've been
writing plugins like that for a while, such as for Apache, and I don't see
any problems with writing vtables or with them making the source hard to
read.

>> I believe the chances of successfully holding to such a decision are
>> zero.  There will be backwards-incompatible changes to function
>> signatures and semantics.  The framework should plan for them from the
>> start.

> Zero because... we can't be expected to not make such a mistake?  Or
> because you don't want that constraint?

I don't think there's any chance that we'll anticipate all the
requirements of a plugin ABI in advance and not have to ever change it in
ways that aren't backward-compatible.  It's not just a matter of mistakes,
although of course those are also not unlikely.  It's a matter of software
evolving over time, which adds new requirements or enforces changes in the
data that's passed through ABIs, and the plugin ABIs are brand new and
don't have the practical experience that the Kerberos ABIs have.

> Note that the above constraint applies only if you want to do anything
> more than reject plugins that don't export the one API version supported
> by the framework.  I.e., if you want to support a variety plugins using
> different API versions, then you must not make backwards-incompatible
> changes to the ABI.

You must not make backwards-incompatible changes to the ABI without
changing the exposed name, whether a vtable or a function, no matter what,
whether you want to support old versions or not.  If you make
backwards-incompatible changes without changing the name, you'll get
segfaults and other serious problems if someone accidentally installs a
module written to the old ABI.

It's just a question of how many differently-named symbols you export and
which the plugin loading code looks for.  You do so by renaming all the
symbols affected by any ABI change so that modules supporting the new ABI
need to provide symbols with a new and different name.  With vtables, this
is straightforwardly encapsulated in the name of the vtable, which should
include the ABI revision.  For pure function names, all the functions the
module exposes should be required to embed the ABI version to make changes
clear and straightforward in the future.

> Note that you could make the same kind of mistakes with a versioned
> v-table scheme too if you're not careful, especially if you allow
> plugins to support more than one version at a time.

Yes.  Either way, one definitely has to be careful.

> For user-land I much prefer (1); formative experience there includes,
> among others, libpam and libgss (see below) (on Solaris).

PAM is, IMO, a bad example here, given that development of the PAM ABI is
completely moribund despite a wealth of fundamental problems with the
existing ABI that should really be fixed.

> On Solaris libgss uses a non-version v-table for old stuff, dlsym() for
> new stuff.  Every time we added functions we had to patch libgss and all
> plugins together.  What a pain.  Granted, with a versioned v-table we'd
> not have to, but now we'd have N versions of the same crappy structure
> lying around, with libgss having to know all N -- that's a lot of
> unnecessary duplication and unnecessary code, plus it's ugly.  So I just
> made it so new additions are dlsym()ed and that's that.

If you believe the most likely change is to add a new function that's
unrelated to all existing functions, and if you think that's likely to
happen frequently, I can definitely see why you would prefer a
function-driven interface.  You otherwise end up with lots of versions of
vtable symbols since you're reving that version each time you introduce a
new function.  I don't believe that's likely to be the case for Kerberos
plugins, though.  I think the most likely changes will be changes to the
signature or behavior of an existing interface, rather than the addition
of a new interface unrelated to existing interfaces in that module.

Of course, to some extent one can transform one problem into the other.
But my guess is that most of the plugins we'll be working with are going
to support a small number of functions representing fundamental
operations.

-- 
Russ Allbery (rra at stanford.edu)             <http://www.eyrie.org/~eagle/>