[krbdev.mit.edu #7132] Reverse DNS happens despite rdns=false

[Jim Carter via RT]

Distro: OpenSuSE 11.4
Version: krb5-1.8.3-16.19.2.x86_64
Library: /usr/lib64/libkrb5.so.3.3
Subroutine: sn2princ()

I have a server whose wild side is on a residential DSL line.  Its IP
address changes randomly and the "A" record for its wild-side name is
updated by dynamic DNS (dyndns.com).  The ISP has PTR records for the
DHCP addresses, which are not under my control.  I want to use Kerberos
for outside clients (my work machine and laptop) to authenticate, using
the dyndns "A" record as the server name.  The current services are
XMPP/GSSAPI (Prosody) and HTTPS/SPNEGO (Apache proxy for SOGo).

Other use-cases for rdns=false are where the clients use a CNAME for the 
server, and where there are multiple "A" records with the same name, for
load sharing among multiple servers.

I've set /etc/krb5.conf [libdefaults] rdns=false on both the server and
the client, but even so, both the server and the client are seen to 
retrieve the PTR record for the IP address, and fail to do Kerberos
authentication (GSSAPI).  If I hijack the ISP's realm and create a
principal for the wild-side canonical name (PTR value), they can 
authenticate, but obviously this is not a satsifactory solution.

The reason RDNS is still happening is that getaddrinfo is being asked
(with the AI_CANONNAME flag) to determine the server's canonical name,
and the subroutine uses it unconditionally.  The included patch makes
that behavior conditional on the rdns configuration variable.

With the patched library (client and server) the client gets a service
ticket for the correct wild-side name, the server knows its correct name
and finds its key in the keytab, the server believes in my service
ticket, and it lets me on.  The Prosody XMPP server had to be hacked to
use a configured name rather than inferring it from gethostname() which
gives the name on the local LAN.  I think Apache's mod_kerb needs the
same treatment but I haven't finished the hack yet.

The existing conditionalized reverse lookup may actually not be needed,
since getaddrinfo has already done it (per AI_CANONNAME if given), but I
didn't mess with code that wasn't actually broken.

If a string representation of a numeric IP address is passed in, we want
to always get the canonical name.  The test I used to recognize a 
numeric address is kind of lameass; it would consider deadbeef.adb.be
to be numeric.  (As well as fe80::210:60ff:fe15:85f4 as it should.)  If
someone knows a neat test for a numeric IP, this is where it's needed.

James F. Carter          Voice 310 825 2897    FAX 310 206 6673
UCLA-Mathnet;  6115 MSA; 405 Hilgard Ave.; Los Angeles, CA, USA 90095-1555
Email: jimc at math.ucla.edu  http://www.math.ucla.edu/~jimc (q.v. for PGP key)

The patch:

If rdns (reverse DNS lookup) is false, use the caller-provided server 
name and head off getaddrinfo() from looking up the PTR record to get 
the server's canonical name.  Except if given a numerical IP address.

Index: src/lib/krb5/os/sn2princ.c
===================================================================
--- src/lib/krb5/os/sn2princ.c.orig	2012-05-10 19:47:03.177952712 -0700
+++ src/lib/krb5/os/sn2princ.c	2012-05-14 13:08:19.954180561 -0700
@@ -67,6 +67,7 @@
  {
      char **hrealms, *realm, *remote_host;
      krb5_error_code retval;
+    int use_rdns, numeric_ip;
      register char *cp;
      char localname[MAXHOSTNAMELEN];

@@ -95,6 +96,12 @@
              struct addrinfo *ai, hints;
              int err;
              char hnamebuf[NI_MAXHOST];
+                /* Always use RDNS if hostname is numeric.  Will getaddrinfo
+                   return AI_NUMERICHOST, so this lameass test is not needed?
+                   The man page doesn't say so.  (Example: deadbeef.aed.be) */
+            use_rdns = maybe_use_reverse_dns(context, DEFAULT_RDNS_LOOKUP) ||
+                (strspn(hostname, "0123456789.:abcdefABCDEF") == 
+                strlen(hostname));

              /* Note that the old code would accept numeric addresses,
                 and if the gethostbyaddr step could convert them to
@@ -108,7 +115,12 @@

              memset(&hints, 0, sizeof(hints));
              hints.ai_family = AF_INET;
-            hints.ai_flags = AI_CANONNAME|AI_ADDRCONFIG;
+                /*
+                 * In DNS, the canonical name is what the PTR record says.
+                 * If we suppress RDNS, we want to see our CNAME, not the
+                 * canonical name, and can do without an unwanted PTR lookup.
+                 */
+            hints.ai_flags = AI_ADDRCONFIG | (use_rdns ? AI_CANONNAME : 0);
          try_getaddrinfo_again:
              err = getaddrinfo(hostname, 0, &hints, &ai);
              if (err) {
@@ -122,13 +134,13 @@
                  }
                  return KRB5_ERR_BAD_HOSTNAME;
              }
-            remote_host = strdup(ai->ai_canonname ? ai->ai_canonname : hostname);
+            remote_host = strdup((use_rdns && ai->ai_canonname) ? ai->ai_canonname : hostname);
              if (!remote_host) {
                  freeaddrinfo(ai);
                  return ENOMEM;
              }

-            if (maybe_use_reverse_dns(context, DEFAULT_RDNS_LOOKUP)) {
+            if (use_rdns) {
                  /*
                   * Do a reverse resolution to get the full name, just in
                   * case there's some funny business going on.  If there