Kerberos cross-realms issue

[Ram Chander]

Hi,

Am trying to copy hbase table to remote cluster via kerberos cross-realms.
But its failing with below error.  Below is krb5.conf

 Has anyone got it working ? Any pointers would be helpful.

 # hbase org.apache.hadoop.hbase.
mapreduce.CopyTable  --peer.adr=zoo.stage.domain2.com:2181:/hbase snap1

 13/07/17 16:30:41 INFO zookeeper.ClientCnxn: Opening socket connection to
server host.stage.domain2.com/10.75.208.25:2181. Will attempt to
SASL-authenticate using Login Context section 'Client'
13/07/17 16:30:41 INFO zookeeper.ClientCnxn: Socket connection established
to host.stage.domain2.com/10.75.208.25:2181, initiating session
13/07/17 16:30:41 INFO zookeeper.ClientCnxn: Session establishment complete
on server host.stage.domain2.com/10.75.208.25:2181, sessionid =
0x13fcd8a987e0034, negotiated timeout = 120000
*13/07/17 16:30:41 ERROR client.ZooKeeperSaslClient: An error:
(java.security.PrivilegedActionException:
javax.security.sasl.SaslException: GSS initiate failed [Caused by
GSSException: No valid credentials provided (Mechanism level: Fail to
create credential. (63) - No service creds)]) occurred when evaluating
Zookeeper Quorum Member's  received SASL token. Zookeeper Client will go to
AUTH_FAILED state.
13/07/17 16:30:41 ERROR zookeeper.ClientCnxn: SASL authentication with
Zookeeper Quorum member failed: javax.security.sasl.SaslException: An
error: (java.security.PrivilegedActionException:
javax.security.sasl.SaslException: GSS initiate failed [Caused by
GSSException: No valid credentials provided (Mechanism level: Fail to
create credential. (63) - No service creds)]) occurred when evaluating
Zookeeper Quorum Member's  received SASL token. Zookeeper Client will go to
AUTH_FAILED state.*
13/07/17 16:30:41 WARN zookeeper.ZKUtil: hconnection-0x13fcd8a987e0034
Unable to set watcher on znode /hbase/master
org.apache.zookeeper.KeeperException$AuthFailedException: KeeperErrorCode =
AuthFailed for /hbase/master

* # cat /etc/krb5.conf*

 [libdefaults]
        default_realm = STAGE.COLO1.DOMAIN.COM<http://stage.colo1.domain.com/>
        allow_weak_crypto = false
        dns_fallback = true
        dns_lookup_realm = true
        dns_lookup_kdc = true

        krb4_config = /etc/krb.conf
        krb4_realms = /etc/krb.realms
        kdc_timesync = 1
        ccache_type = 4
        forwardable = true
        proxiable = true
        ticket_lifetime = 10h
        renew_lifetime = 7d

        v4_instance_resolve = false
        v4_name_convert = {
                host = {
                        rcmd = host
                        ftp = ftp
                }
                plain = {
                        something = something-else
                }
        }
        fcc-mit-ticketflags = true

 [realms]
        STAGE.COLO1.DOMAIN.COM <http://stage.colo1.domain.com/> = {
                kdc = host.colo1.domain.com
                admin_server = host.colo1.domain.com
                max_life = 72h
                max_renewable_life = 8d

        }
        STAGE.COLO2.DOMAIN.COM <http://stage.colo2.domain.com/> = {
                kdc = host.colo2.domain.com
                admin_server = host.colo2.domain.com
                max_life = 72h
                max_renewable_life = 8d

        }
[domain_realm]
        .colo1.domain.com =
STAGE.COLO1.DOMAIN.COM<http://stage.colo1.domain.com/>
        colo1.domain.com =
STAGE.COLO1.DOMAIN.COM<http://stage.colo1.domain.com/>
        stage.colo1.domain.com =
STAGE.COLO1.DOMAIN.COM<http://stage.colo1.domain.com/>
        .stage.colo1.domain.com =
STAGE.COLO1.DOMAIN.COM<http://stage.colo1.domain.com/>

        .colo2.domain.com =
STAGE.COLO2.DOMAIN.COM<http://stage.colo2.domain.com/>
        colo2.domain.com =
STAGE.COLO2.DOMAIN.COM<http://stage.colo2.domain.com/>
        stage.colo2.domain.com =
STAGE.COLO2.DOMAIN.COM<http://stage.colo2.domain.com/>
        .stage.colo2.domain.com =
STAGE.COLO2.DOMAIN.COM<http://stage.colo2.domain.com/>
[capaths]
        STAGE.COLO1.DOMAIN.COM <http://stage.colo1.domain.com/> = {
                STAGE.COLO2.DOMAIN.COM <http://stage.colo2.domain.com/>= .
        }
        STAGE.COLO2.DOMAIN.COM <http://stage.colo2.domain.com/> = {
                STAGE.COLO1.DOMAIN.COM <http://stage.colo1.domain.com/> = .
        }

[login]
        krb4_convert = true
        krb4_get_tickets = false
[logging]
        kdc = FILE:/var/log/kerberos/krb5kdc.log
        admin_server = FILE:/var/log/kerberos/kadmin.log
        default = FILE:/var/log/kerberos/krb5lib.log



*Hbase Ver*:  0.94.2+202-1.cdh4.2.0.p0.11~squeeze-cdh4.2.0
*Kerberos Ver*: 5


Regards,
Ram