Hello Joseph,<br>I really appreciate your words. I am pleased that the load balancer will be put to good use.<br><br>I have written my masters thesis on this topic and it is very near completion. The final copy is due within the next two weeks, after which I will post a PDF to this list for everyone to view. Due to some onerous requirements, the document is over 50 pages. I hope to create an abridged version soon.<br>
<br>In the meantime, ELB is fully operational (just like the 2nd death star). I hope to clean up the visualizer next week so others can use it. Justin has already given me useful suggestions that I simply need to implement.<br>
<br>Here are some graphs that I've been toying with for the thesis experiment:<br><a href="http://dl.dropbox.com/u/224960/sc/index.html">http://dl.dropbox.com/u/224960/sc/index.html</a><br><br>Best,<br>Rajat<br><br><div class="gmail_quote">
On Mon, Jan 31, 2011 at 4:09 AM, Kyeong Soo (Joseph) Kim <span dir="ltr"><<a href="mailto:kyeongsoo.kim@gmail.com">kyeongsoo.kim@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
Hello Rajat,<div><br></div><div>I am very interested in your work on the elastic load balancing; I do remember that you posted some graphs on early results in the past and that you were working on your MSc thesis.</div><div>
<br></div><div>In fact, this new feature will be critical for my current research requiring about 3~400 independent simulation runs and I do highly appreciate your great contribution to the StarCluster.</div><div><br></div>
<div>By the way, I wonder whether you have published your work in any conferences/journals yet.</div><div><br></div><div>Regards,</div><div>Joseph</div><div>--<br>Kyeong Soo (Joseph) Kim, Ph.D.<br>Senior Lecturer in Networking<br>
Room 112, Digital Technium<br>Multidisciplinary Nanotechnology Centre, College of Engineering<br>Swansea University, Singleton Park, Swansea SA2 8PP, Wales UK<br>TEL: +44 (0)1792 602024<br>EMAIL: <a href="http://k.s.kim_at_swansea.ac.uk" target="_blank">k.s.kim_at_swansea.ac.uk</a><br>
HOME: <a href="http://iat-hnrl.swan.ac.uk/" target="_blank">http://iat-hnrl.swan.ac.uk/</a> (group)<br> <a href="http://iat-hnrl.swan.ac.uk/%7Ekks/" target="_blank">http://iat-hnrl.swan.ac.uk/~kks/</a> (personal)<div>
<div></div><div class="h5"><br>
<br><br><div class="gmail_quote">On Fri, Jan 28, 2011 at 6:31 PM, Rajat Banerjee <span dir="ltr"><<a href="mailto:rbanerj@fas.harvard.edu" target="_blank">rbanerj@fas.harvard.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div class="gmail_quote">Hi Archie,<div>Yes, there is ELB built into the latest releases of StarCluster. I wrote it, so feel free to write me (+ the list) with any questions.</div><div><br></div><div>The docs on</div><div>
<a href="http://web.mit.edu/stardev/cluster/docs/index.html" target="_blank">http://web.mit.edu/stardev/cluster/docs/index.html</a></div>
<div><br></div><div>haven't been updated in a while. There is a documentation page on starcluster in the code base, see </div><div>/starcluster/StarCluster/docs/sphinx/load_balancer.rst</div><div><br></div><div>That doc should have all of the information you need, and is readable in plain text.</div>
<div><br></div><div>Typically, this is how I fire up the load balancer:</div><div>starcluster bal <cluster_tag> -m <MAX_NODES you want> -n <MIN_NODES you want></div><div><br></div><div>It will poll the cluster every 60 seconds and make decisions. The decisions are described in load_balancer.rst. There is a visualizer which makes 6 graphs with matplotlib to show you how many nodes are working, how many jobs are running, queued, avg load, etc, but the visualizer still needs a little bit of work.</div>
<div><br></div><div>Hope that helps, and feel free to send back questions.</div><div>Rajat Banerjee</div><div><br><br><div class="gmail_quote"><div><div></div><div>On Fri, Jan 28, 2011 at 12:29 PM, <span dir="ltr"><<a href="mailto:starcluster-request@mit.edu" target="_blank">starcluster-request@mit.edu</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;"><div><div></div><div>Send StarCluster mailing list submissions to<br>
<a href="mailto:starcluster@mit.edu" target="_blank">starcluster@mit.edu</a><br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="http://mailman.mit.edu/mailman/listinfo/starcluster" target="_blank">http://mailman.mit.edu/mailman/listinfo/starcluster</a><br>
or, via email, send a message with subject or body 'help' to<br>
<a href="mailto:starcluster-request@mit.edu" target="_blank">starcluster-request@mit.edu</a><br>
<br>
You can reach the person managing the list at<br>
<a href="mailto:starcluster-owner@mit.edu" target="_blank">starcluster-owner@mit.edu</a><br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of StarCluster digest..."<br>
<br></div></div>Today's Topics:<br>
<br>
1. Starcluster and elastic load balancing (Archie Russell)<div><div></div><div><br>
<br><br>---------- Forwarded message ----------<br>From: Archie Russell <<a href="mailto:archier@gmail.com" target="_blank">archier@gmail.com</a>><br>To: <a href="mailto:starcluster@mit.edu" target="_blank">starcluster@mit.edu</a><br>
Date: Thu, 27 Jan 2011 11:40:00 -0800<br>
Subject: [StarCluster] Starcluster and elastic load balancing<br><br><div>Hi,</div><div><br></div><div>Online it says Starcluster has <span style="font-family: 'Lucida Grande',Verdana,'Bitstream Vera Sans',Arial,sans-serif; font-size: 13px; color: rgb(51, 51, 51); line-height: 19px;">Elastic Load Balancing</span> built into the latest code</div>
<div>version at Github. How would I go about using this? How does it work, e.g.</div><div>when does it fire up new nodes and when does it shut them down?</div><div><br></div><div>Thanks,</div><div>Archie</div>
<br></div></div>_______________________________________________<br>
StarCluster mailing list<br>
<a href="mailto:StarCluster@mit.edu" target="_blank">StarCluster@mit.edu</a><div><br>
<a href="http://mailman.mit.edu/mailman/listinfo/starcluster" target="_blank">http://mailman.mit.edu/mailman/listinfo/starcluster</a><br>
<br></div></blockquote></div><br></div>
</div><br>
<br>_______________________________________________<br>
StarCluster mailing list<br>
<a href="mailto:StarCluster@mit.edu" target="_blank">StarCluster@mit.edu</a><br>
<a href="http://mailman.mit.edu/mailman/listinfo/starcluster" target="_blank">http://mailman.mit.edu/mailman/listinfo/starcluster</a><br>
<br></blockquote></div><br></div></div></div>
</blockquote></div><br>