com_err reunification

[Marcus Watts]

Theodore Tso <tytso at mit.edu> had replied to Russ:
> > I think the OpenAFS com_err adds locking in yet another different way, and
> > I'm not sure if something that just relies on pthreads would be sufficient
> > in OpenAFS (there's all that LWP stuff still in there, but I don't know
> > how it influences this problem).
> 
> Well, that would depend on whether or not LWP and pthreads can coexist
> in the same address space, I would think.
> 
> 						- Ted

In AFS, LWP & pthreads are mutually exclusively, and you *have*
to choose at compile-time.  If you define AFS_PTHREAD_ENV, you get
"pthread" compatible structures (for things like rx connections)
with pthread locks, and you then link against libafs{rpc,auth}.{a,so.*}
and -lpthread.  In this environment, libcom_err worries about locking.
The locking only protects the linked list; the "buffer" variable
is still global so botched "unknown error" behavior is possible.

If you don't define AFS_PTHREAD_ENV, you get LWP compatible structures,
and at link time you supply a bunch of libraries ... librx.a ...
liblwp.a libcom_err.a and not including -lpthread.  LWP is not
(normally) preemptive, so fine grain locking issues disappear including
those associated with libcom_err, and so the locking code isn't even
present for this case.

There are macros, LOCK_ET_LIST and UNLOCK_ET_LIST that control this
locking behavior in the AFS source, which either compile into
pthread_mutex operations or into nothing.
	[ I've got similar locks in my com_err implementation,
	except I just did "#if 1" to define these.  A smarter
	version might use configuration logic to turn this on. ]

If you mix thread libraries, particularly lwp code in a pthread world, it
will not work right.  Sometimes it works well enough to fool the
person responsible into deciding it's the rest of AFS that's broken.

				-Marcus