Multiple KDC's realm heuristic for KRB5CCNAME=DIR:/tmp/mydir/ ccache not working

Wed Jul 25 15:04:01 EDT 2018

In order to support my requirements, I need to call gss_acquire_cred or gss_acquire_cred_from with a unique keytab (not /etc/krb5.keytab), one for each KDC.  I'd like to use the automatic ccache creation that gss_acquire_cred_* does.   gss_acquire_cred is failing with a custom keytab location/name. 
http://web.mit.edu/~kerberos/krb5-latest/doc/appdev/gssapi.html
"If the krb5 mechanism acquires initial tickets using the default client keytab, the resulting tickets will be stored in the default cache or collection, and will be refreshed by future calls togss_acquire_cred as they approach their expire time."
Seems gss_acquire_cred only works when /etc/krb5.keytab is present.   

I've tried these:export KRB5_KTNAME=/opt/development/spgw/spgw-gssapi/GSSAPIMemory/spgateway_icsynergy_net.keytabsetenv("KRB5_KTNAME", "/opt/development/spgw/spgw-gssapi/GSSAPIMemory/spgateway_icsynergy_net.keytab", 1)
krb5_gss_register_acceptor_identity("/opt/development/spgw/spgw-gssapi/GSSAPIMemory/spgateway_icsynergy_net.keytab");

Results in:[8053] 1532545049.921505: Retrieving host/gw.icsynergy.info at ICSYNERGY.NET from FILE:/opt/development/spgw/spgw-gssapi/GSSAPIMemory/spgateway_icsynergy_net.keytab (vno 0, enctype 0) with result: 0/Success[8053] 1532545049.921506: Retrieving host/gw.icsynergy.info at ICSYNERGY.NET from FILE:/etc/krb5.keytab (vno 0, enctype 0) with result: 2/Key table file '/etc/krb5.keytab' not foundgss_acquire_cred:851968 - Unspecified GSS failure.  Minor code may provide more informationgss_acquire_cred:0 - Unknown error
Where as:$ sudo cp spgateway_icsynergy_net.keytab /etc/krb5.keytab
Results in:[15550] 1532545264.591459: Retrieving host/gw.icsynergy.info at ICSYNERGY.NET from FILE:/opt/development/spgw/spgw-gssapi/GSSAPIMemory/spgateway_icsynergy_net.keytab (vno 0, enctype 0) with result: 0/Success[15550] 1532545264.591460: Retrieving host/gw.icsynergy.info at ICSYNERGY.NET from FILE:/etc/krb5.keytab (vno 0, enctype 0) with result: 0/Success[15550] 1532545264.591461: Getting initial credentials for host/gw.icsynergy.info at ICSYNERGY.NET[15550] 1532545264.591462: Looked up etypes in keytab: des-cbc-crc, des, des-cbc-crc, rc4-hmac, aes256-cts, aes128-cts[15550] 1532545264.591464: Sending unauthenticated request[15550] 1532545264.591465: Sending request (212 bytes) to ICSYNERGY.NET[15550] 1532545264.591466: Resolving hostname icsynergy.net[15550] 1532545264.591467: Sending initial UDP request to dgram 192.168.0.175:88[15550] 1532545264.591468: Received answer (204 bytes) from dgram 192.168.0.175:88[15550] 1532545264.591469: Sending DNS URI query for _kerberos.ICSYNERGY.NET.[15550] 1532545264.591470: No URI records found[15550] 1532545264.591471: Sending DNS SRV query for _kerberos-master._udp.ICSYNERGY.NET.[15550] 1532545264.591472: Sending DNS SRV query for _kerberos-master._tcp.ICSYNERGY.NET.[15550] 1532545264.591473: No SRV records found[15550] 1532545264.591474: Response was not from master KDC[15550] 1532545264.591475: Received error from KDC: -1765328359/Additional pre-authentication required[15550] 1532545264.591478: Preauthenticating using KDC method data[15550] 1532545264.591479: Processing preauth types: 16, 15, 19, 2[15550] 1532545264.591480: Selected etype info: etype aes256-cts, salt "ICSYNERGY.NEThostgw.icsynergy.info", params ""[15550] 1532545264.591481: Retrieving host/gw.icsynergy.info at ICSYNERGY.NET from FILE:/etc/krb5.keytab (vno 0, enctype aes256-cts) with result: 0/Success[15550] 1532545264.591482: AS key obtained for encrypted timestamp: aes256-cts/7DFF[15550] 1532545264.591484: Encrypted timestamp (for 1532545264.807742): plain 301AA011180F32303138303732353139303130345AA10502030C533E, encrypted D61656A4F25F462A6FA7A0A1E278ACD80B7EAB042A3104F75EFDBE4C714EA4DA724B084B5DB684330DBD87C6E75B725E73D9DB8B47D553DC[15550] 1532545264.591485: Preauth module encrypted_timestamp (2) (real) returned: 0/Success[15550] 1532545264.591486: Produced preauth for next request: 2[15550] 1532545264.591487: Sending request (292 bytes) to ICSYNERGY.NET[15550] 1532545264.591488: Resolving hostname icsynergy.net[15550] 1532545264.591489: Sending initial UDP request to dgram 192.168.0.175:88[15550] 1532545264.591490: Received answer (98 bytes) from dgram 192.168.0.175:88[15550] 1532545264.591491: Sending DNS URI query for _kerberos.ICSYNERGY.NET.[15550] 1532545264.591492: No URI records found[15550] 1532545264.591493: Sending DNS SRV query for _kerberos-master._udp.ICSYNERGY.NET.[15550] 1532545264.591494: Sending DNS SRV query for _kerberos-master._tcp.ICSYNERGY.NET.[15550] 1532545264.591495: No SRV records found[15550] 1532545264.591496: Response was not from master KDC[15550] 1532545264.591497: Received error from KDC: -1765328332/Response too big for UDP, retry with TCP[15550] 1532545264.591498: Request or response is too big for UDP; retrying with TCP[15550] 1532545264.591499: Sending request (292 bytes) to ICSYNERGY.NET (tcp only)[15550] 1532545264.591500: Resolving hostname icsynergy.net[15550] 1532545264.591501: Initiating TCP connection to stream 192.168.0.175:88[15550] 1532545264.591502: Sending TCP request to stream 192.168.0.175:88[15550] 1532545264.591503: Received answer (1564 bytes) from stream 192.168.0.175:88[15550] 1532545264.591504: Terminating TCP connection to stream 192.168.0.175:88[15550] 1532545264.591505: Sending DNS URI query for _kerberos.ICSYNERGY.NET.[15550] 1532545264.591506: No URI records found[15550] 1532545264.591507: Sending DNS SRV query for _kerberos-master._tcp.ICSYNERGY.NET.[15550] 1532545264.591508: No SRV records found[15550] 1532545264.591509: Response was not from master KDC[15550] 1532545264.591510: Processing preauth types: 19[15550] 1532545264.591511: Selected etype info: etype aes256-cts, salt "ICSYNERGY.NEThostgw.icsynergy.info", params ""[15550] 1532545264.591512: Produced preauth for next request: (empty)[15550] 1532545264.591513: AS key determined by preauth: aes256-cts/7DFF[15550] 1532545264.591514: Decrypted AS reply; session key is: aes256-cts/7FBD[15550] 1532545264.591515: FAST negotiation: unavailable[15550] 1532545264.591516: Initializing FILE:/tmp/krb5cc_1000 with default princ host/gw.icsynergy.info at ICSYNERGY.NET[15550] 1532545264.591517: Storing host/gw.icsynergy.info at ICSYNERGY.NET -> krbtgt/ICSYNERGY.NET at ICSYNERGY.NET in FILE:/tmp/krb5cc_1000[15550] 1532545264.591518: Storing config in FILE:/tmp/krb5cc_1000 for krbtgt/ICSYNERGY.NET at ICSYNERGY.NET: pa_type: 2[15550] 1532545264.591519: Storing host/gw.icsynergy.info at ICSYNERGY.NET -> krb5_ccache_conf_data/pa_type/krbtgt\/ICSYNERGY.NET\@ICSYNERGY.NET at X-CACHECONF: in FILE:/tmp/krb5cc_1000[15550] 1532545264.591520: Storing config in FILE:/tmp/krb5cc_1000 for : refresh_time: 1532563265[15550] 1532545264.591521: Storing host/gw.icsynergy.info at ICSYNERGY.NET -> krb5_ccache_conf_data/refresh_time at X-CACHECONF: in FILE:/tmp/krb5cc_1000[15550] 1532545264.591525: Getting credentials tuser1 at ICSYNERGY.NET -> host/gw.icsynergy.info at ICSYNERGY.NET using ccache FILE:/tmp/krb5cc_1000[15550] 1532545264.591526: Retrieving tuser1 at ICSYNERGY.NET -> host/gw.icsynergy.info at ICSYNERGY.NET from FILE:/tmp/krb5cc_1000 with result: -1765328243/Matching credential not found (filename: /tmp/krb5cc_1000)[15550] 1532545264.591527: Getting credentials host/gw.icsynergy.info at ICSYNERGY.NET -> krbtgt/ICSYNERGY.NET at ICSYNERGY.NET using ccache FILE:/tmp/krb5cc_1000[15550] 1532545264.591528: Retrieving host/gw.icsynergy.info at ICSYNERGY.NET -> krbtgt/ICSYNERGY.NET at ICSYNERGY.NET from FILE:/tmp/krb5cc_1000 with result: 0/Success[15550] 1532545264.591529: Get cred via TGT krbtgt/ICSYNERGY.NET at ICSYNERGY.NET after requesting host/gw.icsynergy.info at ICSYNERGY.NET (canonicalize on)[15550] 1532545264.591530: Generated subkey for TGS request: aes256-cts/B474[15550] 1532545264.591531: etypes requested in TGS request: aes256-cts, aes128-cts, aes256-sha2, aes128-sha2, des3-cbc-sha1, rc4-hmac, camellia128-cts, camellia256-cts[15550] 1532545264.591533: Encoding request body and padata into FAST request[15550] 1532545264.591534: Sending request (2155 bytes) to ICSYNERGY.NET[15550] 1532545264.591535: Resolving hostname icsynergy.net[15550] 1532545264.591536: Initiating TCP connection to stream 192.168.0.175:88[15550] 1532545264.591537: Sending TCP request to stream 192.168.0.175:88[15550] 1532545264.591538: Received answer (1470 bytes) from stream 192.168.0.175:88[15550] 1532545264.591539: Terminating TCP connection to stream 192.168.0.175:88[15550] 1532545264.591540: Sending DNS URI query for _kerberos.ICSYNERGY.NET.[15550] 1532545264.591541: No URI records found[15550] 1532545264.591542: Sending DNS SRV query for _kerberos-master._udp.ICSYNERGY.NET.[15550] 1532545264.591543: Sending DNS SRV query for _kerberos-master._tcp.ICSYNERGY.NET.[15550] 1532545264.591544: No SRV records found[15550] 1532545264.591545: Response was not from master KDC[15550] 1532545264.591546: Decoding FAST response[15550] 1532545264.591547: TGS reply is for tuser1 at ICSYNERGY.NET -> host/gw.icsynergy.info at ICSYNERGY.NET with session key rc4-hmac/D92A[15550] 1532545264.591548: Got cred; 0/Success[15550] 1532545264.591549: Resolving unique ccache of type MEMORY[15550] 1532545264.591550: Initializing MEMORY:aelDQjj with default princ tuser1 at ICSYNERGY.NET[15550] 1532545264.591551: Storing host/gw.icsynergy.info at ICSYNERGY.NET -> krbtgt/ICSYNERGY.NET at ICSYNERGY.NET in MEMORY:aelDQjj[15550] 1532545264.591552: Storing host/gw.icsynergy.info at ICSYNERGY.NET -> krb5_ccache_conf_data/pa_type/krbtgt\/ICSYNERGY.NET\@ICSYNERGY.NET at X-CACHECONF: in MEMORY:aelDQjj[15550] 1532545264.591553: Storing host/gw.icsynergy.info at ICSYNERGY.NET -> krb5_ccache_conf_data/refresh_time at X-CACHECONF: in MEMORY:aelDQjj[15550] 1532545264.591554: Storing config in MEMORY:aelDQjj for : proxy_impersonator: host/gw.icsynergy.info at ICSYNERGY.NET[15550] 1532545264.591555: Storing tuser1 at ICSYNERGY.NET -> krb5_ccache_conf_data/proxy_impersonator at X-CACHECONF: in MEMORY:aelDQjj[15550] 1532545264.591556: Storing tuser1 at ICSYNERGY.NET -> host/gw.icsynergy.info at ICSYNERGY.NET in MEMORY:aelDQjj[15550] 1532545264.591560: Getting credentials tuser1 at ICSYNERGY.NET -> host/gw.icsynergy.info at ICSYNERGY.NET using ccache MEMORY:aelDQjj[15550] 1532545264.591561: Retrieving tuser1 at ICSYNERGY.NET -> host/gw.icsynergy.info at ICSYNERGY.NET from MEMORY:aelDQjj with result: 0/Success[15550] 1532545264.591563: Creating authenticator for tuser1 at ICSYNERGY.NET -> host/gw.icsynergy.info at ICSYNERGY.NET, seqnum 1044310048, subkey rc4-hmac/AE37, session key rc4-hmac/D92A[15550] 1532545264.591568: Retrieving host/gw.icsynergy.info at ICSYNERGY.NET from FILE:/opt/development/spgw/spgw-gssapi/GSSAPIMemory/spgateway_icsynergy_net.keytab (vno 3, enctype rc4-hmac) with result: 0/Success[15550] 1532545264.591569: Decrypted AP-REQ with specified server principal host/gw.icsynergy.info at ICSYNERGY.NET: rc4-hmac/AAC7[15550] 1532545264.591570: AP-REQ ticket: tuser1 at ICSYNERGY.NET -> host/gw.icsynergy.info at ICSYNERGY.NET, session key rc4-hmac/D92A[15550] 1532545264.591571: Negotiated enctype based on authenticator: rc4-hmac[15550] 1532545264.591572: Authenticator contains subkey: rc4-hmac/AE37[15550] 1532545264.591573: Resolving unique ccache of type MEMORY[15550] 1532545264.591574: Initializing MEMORY:0ox1opP with default princ tuser1 at ICSYNERGY.NET[15550] 1532545264.591575: Storing host/gw.icsynergy.info at ICSYNERGY.NET -> krbtgt/ICSYNERGY.NET at ICSYNERGY.NET in MEMORY:0ox1opP[15550] 1532545264.591576: Storing host/gw.icsynergy.info at ICSYNERGY.NET -> krb5_ccache_conf_data/pa_type/krbtgt\/ICSYNERGY.NET\@ICSYNERGY.NET at X-CACHECONF: in MEMORY:0ox1opP[15550] 1532545264.591577: Storing host/gw.icsynergy.info at ICSYNERGY.NET -> krb5_ccache_conf_data/refresh_time at X-CACHECONF: in MEMORY:0ox1opP[15550] 1532545264.591578: Storing config in MEMORY:0ox1opP for : proxy_impersonator: host/gw.icsynergy.info at ICSYNERGY.NET[15550] 1532545264.591579: Storing tuser1 at ICSYNERGY.NET -> krb5_ccache_conf_data/proxy_impersonator at X-CACHECONF: in MEMORY:0ox1opP[15550] 1532545264.591580: Storing tuser1 at ICSYNERGY.NET -> host/gw.icsynergy.info at ICSYNERGY.NET in MEMORY:0ox1opP[15550] 1532545264.591585: Destroying ccache MEMORY:aelDQjj[15550] 1532545264.591589: Getting credentials tuser1 at ICSYNERGY.NET -> HTTP/ics-dc-1.icsynergy.net at ICSYNERGY.NET using ccache MEMORY:0ox1opP[15550] 1532545264.591590: Retrieving tuser1 at ICSYNERGY.NET -> HTTP/ics-dc-1.icsynergy.net at ICSYNERGY.NET from MEMORY:0ox1opP with result: -1765328243/Matching credential not found[15550] 1532545264.591591: Retrieving tuser1 at ICSYNERGY.NET -> host/gw.icsynergy.info at ICSYNERGY.NET from MEMORY:0ox1opP with result: 0/Success[15550] 1532545264.591592: Getting credentials host/gw.icsynergy.info at ICSYNERGY.NET -> HTTP/ics-dc-1.icsynergy.net at ICSYNERGY.NET using ccache MEMORY:0ox1opP[15550] 1532545264.591593: Retrieving host/gw.icsynergy.info at ICSYNERGY.NET -> krbtgt/ICSYNERGY.NET at ICSYNERGY.NET from MEMORY:0ox1opP with result: 0/Success[15550] 1532545264.591594: Starting with TGT for client realm: host/gw.icsynergy.info at ICSYNERGY.NET -> krbtgt/ICSYNERGY.NET at ICSYNERGY.NET[15550] 1532545264.591595: Requesting tickets for HTTP/ics-dc-1.icsynergy.net at ICSYNERGY.NET, referrals on[15550] 1532545264.591596: Generated subkey for TGS request: aes256-cts/CAB1[15550] 1532545264.591597: etypes requested in TGS request: aes256-cts, aes128-cts, aes256-sha2, aes128-sha2, des3-cbc-sha1, rc4-hmac, camellia128-cts, camellia256-cts[15550] 1532545264.591599: Encoding request body and padata into FAST request[15550] 1532545264.591600: Sending request (3855 bytes) to ICSYNERGY.NET[15550] 1532545264.591601: Resolving hostname icsynergy.net[15550] 1532545264.591602: Initiating TCP connection to stream 192.168.0.175:88[15550] 1532545264.591603: Sending TCP request to stream 192.168.0.175:88[15550] 1532545264.591604: Received answer (1622 bytes) from stream 192.168.0.175:88[15550] 1532545264.591605: Terminating TCP connection to stream 192.168.0.175:88[15550] 1532545264.591606: Sending DNS URI query for _kerberos.ICSYNERGY.NET.[15550] 1532545264.591607: No URI records found[15550] 1532545264.591608: Sending DNS SRV query for _kerberos-master._udp.ICSYNERGY.NET.[15550] 1532545264.591609: Sending DNS SRV query for _kerberos-master._tcp.ICSYNERGY.NET.[15550] 1532545264.591610: No SRV records found[15550] 1532545264.591611: Response was not from master KDC[15550] 1532545264.591612: Decoding FAST response[15550] 1532545264.591613: TGS reply is for tuser1 at ICSYNERGY.NET -> HTTP/ics-dc-1.icsynergy.net at ICSYNERGY.NET with session key aes256-cts/CD25[15550] 1532545264.591614: TGS request result: 0/Success[15550] 1532545264.591615: Received creds for desired service HTTP/ics-dc-1.icsynergy.net at ICSYNERGY.NET[15550] 1532545264.591616: Storing tuser1 at ICSYNERGY.NET -> HTTP/ics-dc-1.icsynergy.net at ICSYNERGY.NET in MEMORY:0ox1opP[15550] 1532545264.591618: Creating authenticator for tuser1 at ICSYNERGY.NET -> HTTP/ics-dc-1.icsynergy.net at ICSYNERGY.NET, seqnum 186715939, subkey aes256-cts/1AFF, session key aes256-cts/CD25<<< RUNNING TEST: t_getImpSecurityToken service principal: host/gw.icsynergy.info at ICSYNERGY.NET host: HTTP at ics-dc-1.icsynergy.net user: tuser1 at ICSYNERGY.NET[15550] 1532545264.591623: Destroying ccache MEMORY:0ox1opPSUCCESS service principal: host/gw.icsynergy.info at ICSYNERGY.NET host: HTTP at ics-dc-1.icsynergy.net user: tuser1 at ICSYNERGY.NET!!!GSSAPIMemory END!!!

    On Wednesday, July 25, 2018 9:07 AM, Greg Hudson <ghudson at mit.edu> wrote:

 On 07/24/2018 03:26 PM, Martin Gee wrote:> Would managing KRB5CCNAME 
dynamically via setenv system call be a better
> strategy?  Seems like I basically, need to map the REALM to the 
> appropriate ccache file in a way the gss calles would still work.

That seems like it should work.  You could alternatively use 
gss_acquire_cred_from() to specify the ccache location.  See 
t_credstore.c (in the same place as t_s4u.c) for an example, and use the 
key "ccache".