use of memset and optimization

[Ben Cox]

On Fri, 2002-11-08 at 13:48, John Hascall wrote:
> My final word on this (I promise).
> 
> Sure, we can all invent solutions were it (malloc,use,memset,free)
> happens to "work" w/o volatile.  That's not what's important,
> what's important is there is at least one case where not using
> volatile doesn't work and therefore leaves a small but finite
> vulnerability.
> 
> John

FWIW, I agree, and regret the text I wrote at bottom which, although
true, is not an effective argument toward that point.  (See my
self-refutation below.)

> > On Fri, 2002-11-08 at 12:24, Tom Yu wrote:
> > > john>    1) volatile requirement (as you mention)
> > > john>    2) the signal handler has no way to access that object anyway
> > > john>       (it's only pointed to by "key" which is local to the function)
> > > 
> > > It also occurs to me that even if a pointer to "key" escapes, e.g. by
> > > way of the encrypt() function, which is an external reference, a
> > > signal handler can't have defined behavior if it references "key",
> > > since accesses through a pointer after it has been free()ed are
> > > undefined, and there's no guarantee that the signal handler will be
> > > called before free() invalidates the pointer.  I don't think the
> > > standard requires that a signal only be delivered at sequence points.
> > 
> > If free() is also an external reference, it's possible that the pointer
> > could be remembered by encrypt() (or maybe memset(), for that matter, if
> > the compiler can't prove you're using the system memset()) and forgotten
> > in free(), so the signal handler could know whether it was safe to
> > access.
> > 
> > It's also possible to write signal handlers that *can* work on pointers
> > even after they've been freed.  (For example, ones that know the
> > structure of the malloc arena; consider Purify and its ilk, which can
> > dump memory allocation details on a signal or via timer.)  From an
> > *application* point of view, free()ing a pointer makes it "invalid";
> > from a lower level point of view, calling free() just makes the target
> > memory part of a different data structure.  The fact that the standard
> > cries foul if that pointer gets used doesn't mean that a process that
> > does it is necessarily broken.

To argue against myself: But it also doesn't mean that a compiler which
optimizes away calls that alter that effect violates the standard.

-- Ben