[StarCluster] cluster creation best practice

[Steve Darnell]

Oops… here is a blog post of his regarding how he tested a 10,000 node cluster on EC2: http://blogs.scalablelogic.com/2012/11/running-10000-node-grid-engine-cluster.html

~Steve

From: starcluster-bounces at mit.edu [mailto:starcluster-bounces at mit.edu] On Behalf Of Steve Darnell
Sent: Friday, November 07, 2014 3:42 PM
To: Rajat Banerjee; Ramon Ramirez-Linan
Cc: starcluster at mit.edu
Subject: Re: [StarCluster] cluster creation best practice

Hi Ramon,

As to best practices for starting a large cluster, Rayson has been an advocate for starting a small cluster (5-10 nodes) and then growing the cluster size using the elastic load balancer functionality in StarCluster (http://star.mit.edu/cluster/docs/latest/manual/load_balancer.html).

Here is a blog post of his regarding how he tested a 10,000 node cluster on EC2.

Best regards,
Steve

From: starcluster-bounces at mit.edu<mailto:starcluster-bounces at mit.edu> [mailto:starcluster-bounces at mit.edu] On Behalf Of Rajat Banerjee
Sent: Friday, November 07, 2014 3:23 PM
To: Ramon Ramirez-Linan
Cc: starcluster at mit.edu<mailto:starcluster at mit.edu>
Subject: Re: [StarCluster] cluster creation best practice

Are you requesting a particular type of instance, or in an unusual region or AZ?

I work for AWS and though I have no definitive evidence, I doubt that you'd have difficulty launching most types of instances in any of the big regions, N. Virginia, Oregon, or Ireland.

On Fri, Nov 7, 2014 at 2:49 PM, Ramon Ramirez-Linan <rlinan at navteca.com<mailto:rlinan at navteca.com>> wrote:
Jennifer,

Thanks for your answer although I was aware of all those limits and in fact we requested the limit increased a while ago.

My question was more about how to check if AWS has the resources that I need before asking StartCluster to start creating the cluster.

If I need a 300 nodes cluster and it fails after 250 due to AWS not having enough availability of an instance type, I will still have to pay for the 250 nodes for at least an hour.
After the 250 nodes have been created I have the option of keep trying to add more nodes to complete to 300 but I think to optimize the cost it would be better if I have a way of knowing in advance if AWS is going to have 300 EC2 for me.



PS: I am an AWS Solutions Architect and my company is an APN and reseller partner.


On Thu, Nov 6, 2014 at 12:30 PM, Jennifer Staab <jstaab at cs.unc.edu<mailto:jstaab at cs.unc.edu>> wrote:
AWS does put limits on resources any one AWS account can use (see here <http://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html>  for details).  Specifically see here<http://aws.amazon.com/ec2/faqs/#How_many_instances_can_I_run_in_Amazon_EC2> for limits regarding EC2s. You can make requests to AWS to increase limit bounds. Also note there are certain actions AWS users are not allowed take (see here<http://aws.amazon.com/aup/> ), if you violate any of those policies Amazon has right to deny service to you.

And another thing to consider is the time of the year. AWS started because Amazon wanted to make more use of its resources when they were less than active.  I'm not sure what proportion of AWS resources are still used by Amazon itself, but I would think with holidays approaching the EC2s might be less than available as compared to the start of 2015.

Good Luck.

-Jennifer



On 11/6/14 11:41 AM, Ramon Ramirez-Linan wrote:
I need to create a 300 nodes cluster.

1. if there a way to query AWS in advance to see if they have the 300 EC2 availables? I dont want to find out after 299 nodes that they out of that EC2 type. I am not sure if StarCluster already check for that.

2. Would it be better to create the cluster with less nodes and then add them 20 at a time?

Thanks in advance
Ramon


_______________________________________________

StarCluster mailing list

StarCluster at mit.edu<mailto:StarCluster at mit.edu>

http://mailman.mit.edu/mailman/listinfo/starcluster



_______________________________________________
StarCluster mailing list
StarCluster at mit.edu<mailto:StarCluster at mit.edu>
http://mailman.mit.edu/mailman/listinfo/starcluster

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/pipermail/starcluster/attachments/20141107/1f4ff2ff/attachment-0001.htm