[StarCluster] sge submit host question

[Rayson Ho]

Robert,

I spent some time on this this evening, and basically if you follow the steps I 
listed, PLUS if you set up the security group to allow traffic to port 63231 
(should be very easy if you use the Management Console), then you can have your
local machine submitting jobs to the cluster on EC2.

*However*, I recommend against this, because SGE traffic is not encrypted.

So, the best way is to use the sshmaster command to remotely execute qsub.

You might find that it is not as convenient, as for example you will need to 
setup environment variables, etc on the EC2 machines. (I also did try SSH port
forwarding but the SGE protocol will simply block the connection as the qmaster 
finds it weird that the source IP comes from the host on EC2, while qsub claims
that it is from your local machine.)

If you have a use case that would strongly benefit from SSH port forwarding,
please let me know.


Rayson

=================================
Open Grid Scheduler / Grid Engine
http://gridscheduler.sourceforge.net/

Scalable Grid Engine Support Program
http://www.scalablelogic.com/






----- Original Message -----
From: Justin Riley <jtriley at mit.edu>
To: David Erickson <derickso at stanford.edu>
Cc: Robert Yu <robert.yu at aditazz.com>; starcluster <starcluster at mit.edu>
Sent: Friday, February 24, 2012 4:36 PM
Subject: Re: [StarCluster] sge submit host question

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Yes the environment issue is related to the fact that SSH does not
source /etc/profile by default. For the time being you can get around
this using:

$ starcluster sshmaster mycluster 'source /etc/profile; <command>'

Sourcing /etc/profile should probably be done by default when running
commands using sshmaster/sshnode. I created an issue to keep track of
this:

http://web.mit.edu/star/cluster/issues/84

HTH,

~Justin


On 02/24/2012 03:38 PM, David Erickson wrote:
> On 2/24/2012 12:07 PM, Robert Yu wrote:
>> Hi,
>> 
>> This is more of a best-practice question.
>> 
>> Let's say I am on machine A.  From it, I create a cluster.  I can
>> then ssh into the "master" and launch jobs.  Can this also be
>> done from machine A directly?  I know one can do "starcluster
>> sshmaster -u sgeadmin<clustername>  'some command'" but then I
>> typically run into environment issues.  Is there (simple) a way
>> to make machine A an sge submit host?
>> 
> 
> I ran into this too and found out I had to source 
> /etc/profile.d/sge.sh.  I also am curious if there is a better
> solution, presumably this is an interactive vs noninteractive shell
> issue.
> 
> -D _______________________________________________ StarCluster
> mailing list StarCluster at mit.edu 
> http://mailman.mit.edu/mailman/listinfo/starcluster

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk9IAsIACgkQ4llAkMfDcrmLdQCbBH1KCzDASMcUx0smaFSWU9Gx
HY4An0QKuzBCpEi7ZCuQ06bY9LcRcPzw
=T/Lq
-----END PGP SIGNATURE-----
_______________________________________________
StarCluster mailing list
StarCluster at mit.edu
http://mailman.mit.edu/mailman/listinfo/starcluster