Win2K8 KDC/Linux login A-OK; NFSv4 = permission denied?
Derek Warren
warren at sfu.ca
Fri Aug 24 12:39:56 EDT 2012
Hello all,
We have a test CentOS 6.3 NFSv4 server and client[0], both bound to a Windows 2008 R2 Active Directory domain controller.
We don't appear to be having any issues calling `kinit -k` to get machine credentials[1], or logging in with pam_krb5[2]. (RHEL's `authconfig` took care of all that for us.)
The problem: our test NFS client is unable to list "private" (chmod 700-ed) directories, even if the current user owns the directory in question.
ID mapping works:
[joeuser at nfsclient ~]$ id joeuser
uid=56055(joeuser) gid=6502(domain users) groups=6502(domain users),1000001(BUILTIN\users)
... and we can mount directories with -o sec=none/krb5/krb5i/krb5p:
[joeuser at nfsclient ~]$ sudo mount -t nfs4 -o proto=tcp,port=2049 -o sec=krb5p nfsserver.example.com:/testdir /mnt
[joeuser at nfsclient ~]$ ls -l /mnt
drwx------ 2 user1 domain users 4096 Aug 3 11:43 user1
drwx------ 2 adbinder domain users 4096 Aug 17 15:20 adbinder
drwx------ 2 joeuser domain users 4096 Aug 3 11:43 joeuser
... but we hit the wall here:
[joeuser at nfsclient ~]$ cd /mnt/joeuser
bash: cd: joeuser: Permission denied
At this point, we get a warning from rpc.gssd: "Failed to create krb5 context for user xxx..."[3]
Mailing list archives suggest this is usually a matter of mismatched UIDs/GIDs, but we don't appear to have such a problem--we are happily retrieving[4] identical UID/GID info from our Active Directory DC (W2K8R2) via Samba 3.5 and winbindd on both the server and client.
We are using a least-privileges-required Active Directory bind account[5] for Samba/winbind, but I can reproduce the same issue with a full-on "Domain Admin" bind account.
NFSv4 mounts with sec=sys work fine, but sec=none or sec=krb5/i/p results in the problem described above. World-readable and -executable directories are readily accessed without any problems.
If encryption type mismatches are a problem, neither rpc.svcgssd or rpc.gssd have complained. Our krb5.conf file looks like this: [6]
Does this sound more like a Linux KRB5 client issue or a Win2K8R2 KDC issue? An educated guess or wild speculation from anyone more experienced that me (that's everyone here) would be most welcome.
Many thanks,
--
Derek Warren, IT Services, Research Computing Group, Simon Fraser University
[0] Both hosts are running CentOS 6.3:
$ uname -a ; cat /etc/redhat-release ; rpm -qa | egrep "(samba|winbind|nfs)"
Linux nfsserver 2.6.32-279.2.1.el6.x86_64 #1 SMP Fri Jul 20 01:55:29 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
CentOS release 6.3 (Final)
nfs-utils-1.2.3-26.el6.x86_64
nfs-utils-lib-1.1.5-4.el6.x86_64
samba-client-3.5.10-125.el6.x86_64
samba-common-3.5.10-125.el6.x86_64
samba-winbind-3.5.10-125.el6.x86_64
samba-winbind-clients-3.5.10-125.el6.x86_64
Out of curiosity, I tried the same AD/Winbind/NFSv4 setup on Fedora 17 and had the same results when mounting either -o sec=none or -o sec=krb5/i/p:
$ uname -a ; cat /etc/redhat-release ; rpm -qa | egrep "(samba|winbind|nfs)"
Linux nfsserver-fedora 3.5.2-1.fc17.x86_64 #1 SMP Wed Aug 15 16:09:27 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
Fedora release 17 (Beefy Miracle)
libnfsidmap-0.25-3.fc17.x86_64
libnfsidmap-debuginfo-0.25-3.fc17.x86_64
nfs-utils-1.2.6-3.fc17.x86_64
nfs-utils-debuginfo-1.2.6-3.fc17.x86_64
samba-3.6.6-92.fc17.1.x86_64
samba-client-3.6.6-92.fc17.1.x86_64
samba-common-3.6.6-92.fc17.1.x86_64
samba-winbind-3.6.6-92.fc17.1.x86_64
samba-winbind-clients-3.6.6-92.fc17.1.x86_64
[1] No kinit problems here, we think...
# net ads join createupn="nfs/nfsserver.example.com at AD.EXAMPLE.COM" createcomputer="OU" -U $BINDUSER%$BINDPASSWD
# kinit -k NFSSERVER$
# net ads keytab add nfs
# klist -ke
Keytab name: WRFILE:/etc/krb5.keytab
KVNO Principal
---- --------------------------------------------------------------------------
3 host/nfsserver.example.com at AD.EXAMPLE.COM (des-cbc-crc)
3 host/nfsserver.example.com at AD.EXAMPLE.COM (des-cbc-md5)
3 host/nfsserver.example.com at AD.EXAMPLE.COM (arcfour-hmac)
3 host/NFSSERVER at AD.EXAMPLE.COM (des-cbc-crc)
3 host/NFSSERVER at AD.EXAMPLE.COM (des-cbc-md5)
3 host/NFSSERVER at AD.EXAMPLE.COM (arcfour-hmac)
3 NFSSERVER$@AD.EXAMPLE.COM (des-cbc-crc)
3 NFSSERVER$@AD.EXAMPLE.COM (des-cbc-md5)
3 NFSSERVER$@AD.EXAMPLE.COM (arcfour-hmac)
3 nfs/nfsserver.example.com at AD.EXAMPLE.COM (des-cbc-crc)
3 nfs/nfsserver.example.com at AD.EXAMPLE.COM (des-cbc-md5)
3 nfs/nfsserver.example.com at AD.EXAMPLE.COM (arcfour-hmac)
3 nfs/NFSSERVER at AD.EXAMPLE.COM (des-cbc-crc)
3 nfs/NFSSERVER at AD.EXAMPLE.COM (des-cbc-md5)
3 nfs/NFSSERVER at AD.EXAMPLE.COM (arcfour-hmac)
(same `klist -ke` output on the client, too)
[2] Our clocks are synced and pam_krb5 lets us in:
$ ssh joeuser at nfsclient.example.com
joeuser at nfsclient.example.com's password:
Last login: Fri Aug 24 07:39:42 2012 from nowheresville.example.com
[joeuser at nfsclient ~]$ cd /mnt
/mnt: Permission denied.
[joeuser at nfsclient ~]$ klist -e
Ticket cache: FILE:/tmp/krb5cc_56055_2iPnxk
Default principal: joeuser at AD.EXAMPLE.COM
Valid starting Expires Service principal
08/24/12 08:44:06 08/24/12 18:44:11 krbtgt/AD.EXAMPLE.COM at AD.EXAMPLE.COM
renew until 08/31/12 08:44:06, Etype (skey, tkt): arcfour-hmac, arcfour-hmac
08/24/12 08:44:14 08/24/12 18:44:11 nfs/nfsserver.example.com at AD.EXAMPLE.COM
renew until 08/31/12 08:44:06, Etype (skey, tkt): arcfour-hmac, arcfour-hmac
[3] rpc.gssd complains "Failed to create krb5 context for user with uid xxx..."
nfsclient rpc.gssd[10256]: handling gssd upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt1b)
nfsclient rpc.gssd[10256]: handle_gssd_upcall: 'mech=krb5 uid=56055 enctypes=18,17,16,23,3,1,2 '
nfsclient rpc.gssd[10256]: handling krb5 upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt1b)
nfsclient rpc.gssd[10256]: process_krb5_upcall: service is '<null>'
nfsclient rpc.gssd[10256]: getting credentials for client with uid 56055 for server nfsserver.example.com
nfsclient rpc.gssd[10256]: CC file '/tmp/krb5cc_56055_od8D5s' being considered, with preferred realm 'AD.EXAMPLE.COM'
nfsclient rpc.gssd[10256]: CC file '/tmp/krb5cc_56055_od8D5s'(joeuser at AD.EXAMPLE.COM) passed all checks and has mtime of 1344544593
nfsclient rpc.gssd[10256]: CC file '/tmp/krb5cc_56055_DUEN2I' being considered, with preferred realm 'AD.EXAMPLE.COM'
nfsclient rpc.gssd[10256]: CC file '/tmp/krb5cc_56055_DUEN2I'(joeuser at AD.EXAMPLE.COM) passed all checks and has mtime of 1344543679
nfsclient rpc.gssd[10256]: CC file '/tmp/krb5cc_56055_od8D5s' is our current best match with mtime of 1344544593
nfsclient rpc.gssd[10256]: CC file '/tmp/krb5cc_0' being considered, with preferred realm 'AD.EXAMPLE.COM'
nfsclient rpc.gssd[10256]: CC file '/tmp/krb5cc_0' owned by 0, not 56055
nfsclient rpc.gssd[10256]: using FILE:/tmp/krb5cc_56055_od8D5s as credentials cache for client with uid 56055 for server nfsserver.example.com
nfsclient rpc.gssd[10256]: using environment variable to select krb5 ccache FILE:/tmp/krb5cc_56055_od8D5s
nfsclient rpc.gssd[10256]: creating context using fsuid 56055 (save_uid 0)
nfsclient rpc.gssd[10256]: creating tcp client for server nfsserver.example.com
nfsclient rpc.gssd[10256]: DEBUG: port already set to 2049
nfsclient rpc.gssd[10256]: creating context with server nfs at nfsserver.example.com
nfsclient rpc.gssd[10256]: WARNING: Failed to create krb5 context for user with uid 56055 for server nfsserver.example.com
nfsclient rpc.gssd[10256]: WARNING: Failed to create krb5 context for user with uid 56055 for server nfsserver.example.com
nfsclient rpc.gssd[10256]: doing error downcall
[4] ID mapping problems? Nothing seems wrong here:
nfsserver rpc.idmapd[1455]: Server : (user) id "56055" -> name "joeuser at ad.example.com"
nfsserver rpc.idmapd[1455]: nfsdcb: authbuf=gss/krb5p authtype=group
nfsserver rpc.idmapd[1455]: nfs4_gid_to_name: calling nsswitch->gid_to_name
nfsserver rpc.idmapd[1455]: nfs4_gid_to_name: nsswitch->gid_to_name returned 0
nfsserver rpc.idmapd[1455]: nfs4_gid_to_name: final return value is 0
nfsserver rpc.idmapd[1455]: Server : (group) id "6502" -> name "domain users at ad.example.com"
nfsserver rpc.idmapd[1455]: nfsdcb: authbuf=gss/krb5p authtype=user
nfsserver rpc.idmapd[1455]: nfs4_uid_to_name: calling nsswitch->uid_to_name
nfsserver rpc.idmapd[1455]: nfs4_uid_to_name: nsswitch->uid_to_name returned 0
nfsserver rpc.idmapd[1455]: nfs4_uid_to_name: final return value is 0
nfsserver rpc.idmapd[1455]: Server : (user) id "0" -> name "root at ad.example.com"
nfsserver rpc.idmapd[1455]: nfsdcb: authbuf=199.60.0.0/255.255.240.0 authtype=group
nfsserver rpc.idmapd[1455]: nfs4_gid_to_name: calling nsswitch->gid_to_name
nfsserver rpc.idmapd[1455]: nfs4_gid_to_name: nsswitch->gid_to_name returned 0
nfsserver rpc.idmapd[1455]: nfs4_gid_to_name: final return value is 0
nfsserver rpc.idmapd[1455]: Server : (group) id "0" -> name "root at ad.example.com"
nfsserver rpc.idmapd[1455]: nfsdcb: authbuf=199.60.0.0/255.255.240.0 authtype=user
nfsserver rpc.idmapd[1455]: nfs4_uid_to_name: calling nsswitch->uid_to_name
nfsserver rpc.idmapd[1455]: nfs4_uid_to_name: nsswitch->uid_to_name returned 0
nfsserver rpc.idmapd[1455]: nfs4_uid_to_name: final return value is 0
[5] Our method for creating a least-privileges-required Samba/winbind bind account on a W2K8R2 DC, taken from HP Samba documentation:
1. Run ADU&C. View menu=>Advanced Features
2. Create a binding account (e.g., 'megabind')
3. Right-click on the relevant OU=>Properties=>Security tab
4. [Advanced]=>[Add...]=>megabind=>[OK]
5. [megabind: This object and all descendant objects...]
=> Write All Properties: allow
=> Create Computer objects: allow
6. [OK]/[OK]/[OK]
[6] http://www.sfu.ca/~warren/krb5.conf.txt
More information about the Kerberos
mailing list