[StarCluster] is mandatory that master instance is the same kind of instance than compute nodes?

[MacMullan, Hugh]

If the master instance type is not of 'PLACEMENT_GROUP_TYPES' (M3_COMPUTE_TYPES + HVM_ONLY_TYPES + HI_IO_TYPES + HI_STORAGE_TYPES), but the nodes are, a placement group IS still created, but master isn't part of it. Excellent! c3.large ($0.105/hr on demaind) is of HVM_ONLY_TYPES, so might be a good alternative at $2.52/day (and it's VERY cheap with spot).

Also: just because the master is 'smaller' doesn't mean you need to disable the queue. StarCluster will give you slots == number of cores. You can manually modify all.q to be cores - 1 … so you have at least one 'free' core. I like that setup for 'loadbalance' setups, so you get SOME work started at least. Also to launch daemons and stuff encapsulated in GE.

Of course if you only have one core to begin with … :)

-Hugh

From: starcluster-bounces at mit.edu [mailto:starcluster-bounces at mit.edu] On Behalf Of Sergio Mafra
Sent: Friday, October 24, 2014 10:05 AM
To: Pablo Escobar
Cc: starcluster at mit.edu
Subject: Re: [StarCluster] is mandatory that master instance is the same kind of instance than compute nodes?

Hi folks,

If you decide to have a small instance to be a master node and big ones to run as slaves, what should happens to the placement group? Does StarCluster put only the slaves (depending on the instance size) on the placement group or none?

All best,

Sergio

On Fri, Oct 24, 2014 at 9:33 AM, Pablo Escobar <pescobar001 at gmail.com<mailto:pescobar001 at gmail.com>> wrote:
Hi Robert,

After reading your email I have seen that in the configuration there are two options about instance types, NODE_INSTANCE_TYPE  and MASTER_INSTANCE_TYPE

So the solution to achieve my desired setup seems to configure the master to not being an exec node and select different type of instance for node and master.

Thanks for your help and sorry for the beginner question...just doing my first steps with starcluster

Pablo.



2014-10-24 13:27 GMT+02:00 Petit III, Robert A. <robert.petit at emory.edu<mailto:robert.petit at emory.edu>>:

Hi Pablo,

If the master is set to not be an exec node, the worker nodes can be a different type.  The situation you describe, (small master instance, large compute instances) is my current set up.

I believe things are different when the master is set as an exec host.

Cheers,
Robert

________________________________
From: starcluster-bounces at mit.edu<mailto:starcluster-bounces at mit.edu> [starcluster-bounces at mit.edu<mailto:starcluster-bounces at mit.edu>] on behalf of Pablo Escobar [pescobar001 at gmail.com<mailto:pescobar001 at gmail.com>]
Sent: Friday, October 24, 2014 5:48 AM
To: starcluster at mit.edu<mailto:starcluster at mit.edu>
Subject: [StarCluster] is mandatory that master instance is the same kind of instance than compute nodes?
Hi,

I have been doing some testing with starcluster and I see that by default every machine is the same type of instance and that the sge_master is also an sge_exec node.

I have read in the documentation that the sge plugin allows to configure the master node to not being an exec node.

The ideal setup for me would be to have the master node being a small instance just running the sge_master and not running jobs and the compute nodes being bigger instances in terms of hardware resources.

is this setup currently possible in starcluster? or is mandatory that every machine is the same kind of instance?

thanks for your help.

Pablo.

________________________________

This e-mail message (including any attachments) is for the sole use of
the intended recipient(s) and may contain confidential and privileged
information. If the reader of this message is not the intended
recipient, you are hereby notified that any dissemination, distribution
or copying of this message (including any attachments) is strictly
prohibited.

If you have received this message in error, please contact
the sender by reply e-mail message and destroy all copies of the
original message (including attachments).


_______________________________________________
StarCluster mailing list
StarCluster at mit.edu<mailto:StarCluster at mit.edu>
http://mailman.mit.edu/mailman/listinfo/starcluster

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/pipermail/starcluster/attachments/20141024/2f843bae/attachment-0001.htm