design choices for a loadable module interface

[Nicolas Williams]

On Tue, May 25, 2010 at 06:06:31PM -0400, Tom Yu wrote:
> We are still working through some design issues related to an improved
> plugin architecture.  We would like some input on what form a loadable
> module interface should take.  This is the interface that would allow
> a caller to do something like dynamically load a shared object at run
> time, e.g. dlopen() on Unix-like systems.  We would not necessarily
> immediately adopt such a loadable module interface for existing
> pluggable interfaces, but would migrate toward it over time.

I assume that by "builtin" you mean "statically linked in".

> Below are several possible alternatives.  Please comment if you have
> strong preferences among them.
> 
> * exported data symbol -- a vtable (a structure of function pointers)
> 
>   # We already use this for preauth and authdata plugins.
>   
>   # Shared object data symbols require extra relocation overhead on
>     some platforms.

Exported data symbols are more complicated under the covers than
exported function symbols.  I don't have my copy of the linkers and
loaders book with me, but if necessary I'll look this up.

I strongly recommend against exported data symbols in an SPI.

> * exported function symbol(s)
> 
>   -- separate symbol for each interface function
> 
>      # Runtime symbol resolution can be slow on some platforms.

But it's one-time.  The framework should memoize the dlsym() calls.

>   -- one function that returns a vtable, possibly parameterized by
>      version identifier
> 
>      # This may still require additional relocation overhead on some
>        platforms.

The function pointers in the vtable should be pointers to either static
or non-exported functions.  Static functions are always supported, but
you'd have to put the whole plugin in one source file...  Non-exported
non-static functions are generally supported, but not necessarily always
(I don't think catering to platforms that don't support this is
necessary, but maybe I'm biased! :)

>      # Alternatively, have the function populate a passed-in
>        caller-allocated vtable, possibly with a sanity check on the
>        structure size.  This can still cause additional relocations,
>        unless the function assigns the entries one at a time (e.g., by
>        doing arithmetic on PLT entries), instead of copying from a
>        prepopulated (private) struture.

V-table versioning is needed in either approach.

Either v-table scheme means having to Write More Code.  Yuck, but I'd
live.

> Should different modules that implement the same interface each export
> a different name?  If so, how would the caller discover the correct
> name to pass to dlsym() or equivalent?

IMO: yes, they should alluse the same symbol names.

> Relatedly, do we want to be able to use the same shared object as both
> a loadable module and as a builtin?

For static linking you should reduce the symbol scope to static and then
resort to a vtable -- an internal dlsym(), effectively.  This might
require concatenating source files and using a C pre-processor symbol to
expand to the keyword 'static'.

Or you might:

> Using different names per module might simplify linking the module as
> a builtin, but the loader for builtin modules would still know how to
> find it.

use different names (possibly made to appear as the same in the source
files by using auto-generated C pre-processor macros to rename the
symbols to the plugin-specific names) and build your own internal
dlsym() on top of that.

I believe you're not the first to have to struggle with this.  You might
look at how others have dealt with this.  I think you'll find that you
can't really avoid having to build a sort of internal dlsym().

I realize you can't ditch static linking for some of your users, so I'll
refrain from offering that as an alternative :)

Nico
--