C99 VLA (=Variable Lengh Array) helper macros... / was: Re: C99 Features

[Nathaniel McCallum]

On Thu, 2015-07-09 at 12:40 -0400, Roland Mainz wrote:
> 
> ----- Original Message -----
> > From: "Nathaniel McCallum" <npmccallum at redhat.com>
> > To: "Roland Mainz" <rmainz at redhat.com>, krbdev at mit.edu
> > Sent: Thursday, July 9, 2015 6:21:50 PM
> > Subject: Re: C99 VLA (=Variable Lengh Array) helper macros... / 
> > was: Re: C99 Features
> > 
> > On Thu, 2015-07-09 at 10:44 -0400, Roland Mainz wrote:
> > > 
> > > ----- Original Message -----
> > > > From: "Roland Mainz" <rmainz at redhat.com>
> > > > To: krbdev at mit.edu
> > > > Sent: Tuesday, July 7, 2015 6:33:01 PM
> > > > Subject: Re: C99 Features
> > > > 
> > > > 
> > > > 
> > > > ----- Original Message -----
> > > > > From: "Nathaniel McCallum" <npmccallum at redhat.com>
> > > > > To: krbdev at mit.edu
> > > > > Sent: Tuesday, June 16, 2015 3:06:39 AM
> > > > > Subject: C99 Features
> > > > > 
> > > > > It has been 16 years. GCC has had support for many C99 
> > > > > features
> > > > > for a
> > > > > LongTimeNowTM. Clang has had them since the beginning. Clang 
> > > > > also
> > > > > now
> > > > > has official Windows builds (as well as many other 
> > > > > platforms).
> > > > > 
> > > > > Of course, MSVC still lags behind. However, they have started
> > > > > implementing features, including the following since MSVC 
> > > > > 2013
> > > > > [1]:
> > > > >  * _Bool
> > > > >  * Compound literals
> > > > >  * Designated initializers
> > > > >  * Mixing declarations with code
> > > > > 
> > > > > MS has also implemented most of the C99 libraries[2] and has
> > > > > already
> > > > > announced complete support for the C99 standard library in 
> > > > > MSVC
> > > > > 2015[3]. One of the big elephants in the room is VLAs. MS is
> > > > > unlikely
> > > > > to ever support them. But, it is also MS's stated policy to 
> > > > > only
> > > > > incorporate C features that are required for C++[4]. I'm not 
> > > > > sure
> > > > > that
> > > > > MIT should hold back support for new C features because MS 
> > > > > only
> > > > > wants
> > > > > to ship a C++ compiler.
> > > > > 
> > > > > So how about it? Can MIT start using C99 features? Even a 
> > > > > subset
> > > > > of the
> > > > > features would be helpful. I particularly care about all the
> > > > > above C99
> > > > > features that MS implemented plus VLAs. The latter, in
> > > > > particular, can
> > > > > eliminate a lot of heap allocations.
> > > > 
> > > > 1. Using |bool| would be a very good idea because it allows a 
> > > > lot
> > > > of
> > > > optimizations with low optimizer settings. There is a possible 
> > > > risk
> > > > of
> > > > namespace clash with |krb5_boolean| so I suggest to use a new
> > > > define like
> > > > > krb5_std_bool| if we want it in public APIs (which we should 
> > > > > do
> > > > > (question
> > > > for myself... did the ISO C spec define a size for |bool| ?))
> > > > 2. IMHO we should use |restrict| where possible (e.g. crypto,
> > > > string-heavy
> > > > code etc.) to squeeze out some performance
> > > > 3. VLAs can be tricky, but usually aren't a big problem if used
> > > > with care
> > > > (I've been there with Solaris's OS/Net long ago and it was a 
> > > > *pain*
> > > > politically-wise and getting all tools fixed, but these should 
> > > > be
> > > > issues if
> > > > the past). Main issue is to define an upper per-allocation 
> > > > limit
> > > > which can
> > > > be allocated via VLAs and then switch to |malloc()| if the
> > > > allocation is
> > > > larger than that limit (typically |getpagesize()*8| was 
> > > > considered
> > > > a "safe"
> > > > limit in Solaris OS/Net). If I recall it correctly the 
> > > > following
> > > > code should
> > > > handle the issues correctly (except that it should use the ISO 
> > > > C99
> > > > feature
> > > > test macros to test whether VLAs are available (this will cover 
> > > > the
> > > > "limitations" of the Microsoft compiler)):
> > > > -- snip --
> > > [snip]
> > > > -- snip --
> > > 
> > > Attached (as "vla.c.txt") is an updated version which fixes the
> > > reported issues:
> > > - Added support to compile with pre-C99 compilers
> > > - Added support for C11 VLA feature test macro |__STDC_NO_VLA__|
> > > - Fix multiple evaluations of the |size| argument in
> > > > VLA_ALLOC_ALLOCATE()|
> > > - Make testcase more troublesome for compilers and valgrind
> > > - Add more comments how stuff works
> > > - Attach code instead of pasting it inline to prevent the WebMail
> > > software from messing with the code
> > 
> > If you have to free the VLA, you've just lost half the benefit. It
> > would be better to fall back on alloca().
> 
> We don't have to free the VLA, the |VLA_ALLOC_FREE()| macro is there 
> because (see code - http://mailman.mit.edu/pipermail/krbdev/attachmen
> ts/20150709/4ec3a6c8/attachment.txt) we fall-back to |malloc()| if 
> the requested size is "too large" (this guards against accidental 
> stack exhaustion by a single VLA allocation or malicious attempts to 
> access "random" memory areas beyond the stack space). We directly 
> call |free()| because if we get the memory from a VLA array the 
> pointer which tracks |malloc()|'ed memory is |NULL| and POSIX 
> |free()| handles |free(NULL);| as no-op (we could change this to |if 
> (__builtin_expect(memory_from_malloc, 0)) free(memory_from_malloc);| 
> for the sake of making it easier for the compiler's loop optimiser 
> (if loops are involved at the same C block level)). 

I understand. But it means that you have to call VLA_ALLOC_FREE() at
all. This should not happen.

> > But seriously, I'd prefer this discussion not get hung upon VLAs. I
> > like them. But I care about other C99isms like compound literals 
> > much
> > more.
> 
> Agreed.... and |restrict|&&co. (plus gcc's |__builtin_prefetch()| 
> (see https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html#index
> -g_t_005f_005fbuiltin_005fprefetch-4067) for crypto code) are 
> interesting too.
> 
> ----
> 
> Bye,
> Roland
>