<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">I've made some benchmarks some weeks ago about large cluster deployment with StarCluster. <div><br></div><div>I reported the result in this mailing list, but it could be interesting to share it again. </div><div><br></div><div>It turned out that StarCluster takes 15 minutes every 50 nodes to be launched (micro instances). Something like: </div><div><br></div><div> - 50 nodes: boot time ~ 15 minutes </div><div> - 100 nodes: ~ 30 minutes </div><div> - 200 nodes: ~ 1 hour </div><div><br></div><div><div>This linear progression made me think that StarCluster uses a serial mechanism to start the instances. But it is only a speculation and cannot say more. </div><div><br></div><div>Anyway solving this problem would be a huge improvement for StarCluster. </div></div><div><br></div><div><br></div><div>Cheers,</div><div>Paolo</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br><div><div>On Dec 21, 2011, at 12:25 PM, Sumita Sinha wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">Hello Justin,<div><br></div><div>I restarted a 200 node cluster and this time the step of <b>Installing Sun Grid Engine</b> was taking a lot of time.</div><div><br></div><div>>>> Installing Sun Grid Engine</div><div>
197/199 |///////////////////////////////////////////////////////////////| 98% </div><div><br></div><div>I did a tail to the <b>~/.starcluster/logs/debug.log</b> file and could see the below line getting repeated .</div><div>
<br></div><div>2011-12-21 11:09:06,152 PID: 17885 threadpool.py:136 - DEBUG - unfinished_tasks = 2</div><div><br></div><div>I waited for almost 50 minutes and then had to terminate the cluster.</div><div><br></div><div><br clear="all">
<div><br></div>-- <br>Regards<br>Sumita Sinha<br><br><br>
</div>
_______________________________________________<br>StarCluster mailing list<br><a href="mailto:StarCluster@mit.edu">StarCluster@mit.edu</a><br>http://mailman.mit.edu/mailman/listinfo/starcluster<br></blockquote></div><br></div></body></html>