use of memset and optimization

[Theodore Ts'o]

On Fri, Nov 08, 2002 at 02:03:52AM -0500, Tom Yu wrote:
> >>>>> "Sam" == Sam Hartman <hartmans at MIT.EDU> writes:
> 
> Sam> How does the compiler know that memset has no side effects
> Sam> without cross-function optimization.  I agree that this could be
> Sam> an issue for some compilers, I'm just surprised that gcc is smart
> Sam> enough to optimize this.
> 
> gcc 3.x seems to have an inline memset() implementation if
> optimization is active, so it *knows* all the side effects (only
> writing to the specified memory) of calling memset.  Arguably, this
> constitutes a cross-function optimization.

I did some experimentation on this yesterday when I first saw a report
on this issue on the RISKS list, and with gcc 3.0 and 3.2, the results
I got back were *wierd*.  Using the following program:

#include <stdio.h>
#include <string.h>
#include <sys/types.h>

int encrypt( const void *key )
  {
  puts( key );     /* Normally we'd encrypt here */
  }

int main( int argc, char **argv )  /* Because we can */
  {
  char key[ 16 ];
	  
  strcpy( key, "secretkey" );
  encrypt( key );
  memset( key, 0, sizeof(key) );
  }

I was indeed able to see the memset getting optimized away.  *However*
the effect is dependent on the size of key[].  If key is larger than
16 bytes, it is *not* optimized away.  If it is 16 bytes or smaller,
the memset does indeed disappear.  Presence of other automatic
variables on the stack, either before or after key[], doesn't seem to
make a difference.

So the effect is real, but fortunately, I haven't been able to find
any real life places where this mis-optimization would cause a
problem.  On the other hand, so far the only place that I've looked is
the gpg sources, which is substantially smaller than the krb5
codebase.

						- Ted