Packing pragmas in MIT's gssapi.h header file break ABI compatibility

[Tom Yu]

Simon Wilkinson <simon at sxw.org.uk> writes:

> MIT's gssapi.h header file contains the following
>
> #  if TARGET_OS_MAC
> #    pragma pack(2, push)
> #  endif

These pragmas are in the header for what are probably historical
reasons involving m68k processors.

> This breaks ABI compatibility for callers into the GSSAPI library on  
> Mac OS X - it means that programs which attempt to dynamically locate  
> a suitable GSSAPI library must know how the library they're linking  
> against has been built.

Yes, we know this is a problem.  Unfortunately, as Apple has already
shipped libraries with the "wrong" ABI and programs that link against
them, any change will require binaries compiled against the "wrong"
ABI to recompile against the "right" ABI.  We are open to suggestions
of ways to fix this.

GSS-API is specified as a source-level API, not an ABI.  See Appendix
B of RFC 2744 for some of the issues.  Accordingly, it would appear
that there is currently a de-facto ABI for GSS-API on 64-bit Mac OS X
that uses a structure alignment which is not the compiler default.

> It causes problems with swapping between MIT and Heimdal libraries at  
> link time, and also breaks applications such as OpenSSH which allow  
> the use of many alternate GSSAPI libraries depending on linker  
> parameters.
>
> This problem came to light because it broke both Firefox and  
> Thunderbird on 64bit Mac OS X platforms, which include their own  
> gssapi.h so they can build on build hosts that don't have any GSSAPI  
> providers installed. David Bienvenu from Mozilla Messaging did the leg  
> work in tracking it down.

If they use their own gssapi.h to compile on platforms that have no
GSS installed, perhaps including this ABI exception for Mac OS X is
needed.  Alternatively, as that approach introduces an explicit
platform dependency where there arguably shouldn't be one, Apple could
help to come up with a way to change the ABI on Mac OS X in a way that
doesn't break existing compiled code.