[krbdev.mit.edu #6433] Ticket 5995 (r20586) broke ticket acquistion on Microsoft Windows

[Jeffrey Altman via RT]

Ticket 5995 (r20586) lib/krb5/os/sendto_kdc.c was designed to prevent
socket fd values larger than FD_SETSIZE from being accepted because in
gssapi there are arrays of size FD_SETSIZE that track fd connection state.

Unfortunately, this commit made two errors:

1. It fails to take into account that on Windows, a SOCKET is a form of
HANDLE and it is not an integer value ranging from 0 to FD_SETSIZE-1. 
In fact, on Windows, an FD_SET is of arbitrary size and the number of
file descriptors parameter in select() calls is simply ignored. 

2. Since SOCKETs are not run time library file descriptors,
closesocket() must be used to close the socket and not close().

As a result of the first error, all sockets that are allocated are
larger than FD_SETSIZE and are immediately closed.  It is therefore
impossible to communicate with a KDC.

As a result of the second error, memory is corrupted and random crashes
will occur upon process exit.

A proposed patch:

Index: os/sendto_kdc.c
===================================================================
--- os/sendto_kdc.c     (revision 22114)
+++ os/sendto_kdc.c     (working copy)
@@ -654,12 +654,17 @@
        dprint("socket: %m creating with af %d\n", state->err,
ai->ai_family);
        return -1;              /* try other hosts */
     }
-    if (fd >= FD_SETSIZE) {
-       close(fd);
+#ifndef _WIN32
+       /* Windows sockets are handles, not sequential ints.
+        * FD_SETSIZE is meaningless on Windows.
+        */
+       if (fd >= FD_SETSIZE) {
+       closesocket(fd);
        state->err = EMFILE;
        dprint("socket: fd %d too high\n", fd);
        return -1;
     }
+#endif /* _WIN32 */
     /* Make it non-blocking.  */
     if (ai->ai_socktype == SOCK_STREAM) {
        static const int one = 1;

Please pullup to 1.6 branch when applied.

Jeffrey Altman