<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Thanks Justin. When I asked this question, I had *thought* I was already running the development version, but the Python/Mac setup had thoroughly confused me. Several weeks ago I realized this and upgraded to the GIT version.<div><br></div><div>That has been working very well for me, including the SPOT_BID setting. I've tried the load balancing feature as well, and 3 of the 4 times I tried it, it worked perfectly (not sure what happened the other time). I ended up hacking the loadbalancer code to more aggressively add nodes. I made a few other changes; they're probably not worth contributing, but I'd be glad to if you like. I'll paste one below that might be helpful though, for the functions: "get_running_jobs" "get_queued_jobs" (Lemme know the best way to send a patch if you want).</div><div><br></div><div>Having used it a bunch, the only other feedback I have is that it would be nice if "addnode" took a parameter for *how many* nodes to add, so you could add 10 in one go. E.g. starcluster addnode -n 10</div><div><br></div><div>The doc update and release will be very nice, because the system is already much more capable than the website would make it seem. I have been recommending it to others. Starcluster rocks!</div><div><br></div><div>Ben</div><div><br></div><div><br></div><div>In: StarCluster/starcluster/balancers/sge/__init__.py</div><div><div><br></div><div>I think this requires a change to parse_qstat() as well. The idea is that a job may be "pending", but if it's "state" is "hqw", then it's pending, but it's also waiting for something else to finish (or it's been put on hold manually). Presumably, you don't want to launch new instances for pending jobs that are on hold anyway.</div><div><br></div><div><br></div><div><div><font class="Apple-style-span" face="Courier"> def get_running_jobs(self): </font></div><div><font class="Apple-style-span" face="Courier"> "returns an array of the running jobs, values stored in dictionary" </font></div><div><font class="Apple-style-span" face="Courier"> running = [] </font></div><div><font class="Apple-style-span" face="Courier"> for j in self.jobs: </font></div><div><font class="Apple-style-span" face="Courier"> #if j['job_state'] == u'running': </font></div><div><font class="Apple-style-span" face="Courier"> if j['state'] == u'r': </font></div><div><font class="Apple-style-span" face="Courier"> running.append(j) </font></div><div><font class="Apple-style-span" face="Courier"> return running </font></div><div><font class="Apple-style-span" face="Courier"> </font></div><div><font class="Apple-style-span" face="Courier"> def get_queued_jobs(self): </font></div><div><font class="Apple-style-span" face="Courier"> "returns an array of the queued jobs, values stored in dictionary" </font></div><div><font class="Apple-style-span" face="Courier"> queued = [] </font></div><div><font class="Apple-style-span" face="Courier"> for j in self.jobs: </font></div><div><font class="Apple-style-span" face="Courier"> #if j['job_state'] == u'pending': </font></div><div><font class="Apple-style-span" face="Courier"> if j['state'] == u'qw': </font></div><div><font class="Apple-style-span" face="Courier"> queued.append(j) </font></div><div><font class="Apple-style-span" face="Courier"> return queued </font></div></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br><div><div>On Apr 19, 2011, at 12:51 AM, Justin Riley wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div>Hi Ben/Simms,<br><br>Unfortunately the spot bid is not a supported config option in the current stable version of StarCluster (0.91.2). I've added the ability to specify a SPOT_BID setting within a cluster template in the latest development code if you're interested in testing. This functionality will be available in the 0.92 release. It's taking me forever to release this next version due to other projects at work but I intend to release soon. My apologies to everyone for the extreme delay. At this point I need to update the docs and then it's ready. I plan to have a *much* more frequent incremental release cycle after this next big release comes out.<br><br>~Justin<br><br><br>On Mar 21, 2011, at 5:49 PM, Justin Simms <a href="mailto:justin.simms@gmail.com">justin.simms@gmail.com</a> wrote:<br><br><blockquote type="cite">Ben,<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">As far as I can tell, it seems to be the case that there is no conversion<br></blockquote><blockquote type="cite">from the string type to the float or int type that is expected. The way that<br></blockquote><blockquote type="cite">you have provided your argument is correct.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">jts<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">-----Original Message-----<br></blockquote><blockquote type="cite">From: <a href="mailto:starcluster-bounces@mit.edu">starcluster-bounces@mit.edu</a> [mailto:starcluster-bounces@mit.edu] On<br></blockquote><blockquote type="cite">Behalf Of Benjamin Lambert<br></blockquote><blockquote type="cite">Sent: Monday, March 21, 2011 3:30 PM<br></blockquote><blockquote type="cite">To: <a href="mailto:starcluster@mit.edu">starcluster@mit.edu</a><br></blockquote><blockquote type="cite">Subject: [StarCluster] Setting a spot bid<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Hi StarCluster folks,<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">I just came across StarCluster last week. We've been having some trouble<br></blockquote><blockquote type="cite">with a local physical SGE cluster, and I needed a backup/replacement. I<br></blockquote><blockquote type="cite">made the mistake of trying to set up an existing Ubuntu AMI to work in<br></blockquote><blockquote type="cite">StarCluster, without knowing what I was getting in to. But once I gave up<br></blockquote><blockquote type="cite">on that and started fresh from the default SG Ubuntu 10.04 64-bit image, I<br></blockquote><blockquote type="cite">got things up and running smoothly. Very nice work!<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">I'm looking forward to the features planned for version 0.92, especially<br></blockquote><blockquote type="cite">elastic load balancing, add/remove nodes, and EBS-backed AMIs. Any estimate<br></blockquote><blockquote type="cite">when we might see 0.92?<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">I am having a slight issue with bidding for spot instances. It works just<br></blockquote><blockquote type="cite">fine if I request it through the command line:<br></blockquote><blockquote type="cite">starcluster start --bid=0.50 cluster<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">But, if I put the bid in my .starcluster/config file like so:<br></blockquote><blockquote type="cite">[cluster medium]<br></blockquote><blockquote type="cite">KEYNAME = ...<br></blockquote><blockquote type="cite">SPOT_BID = 0.35<br></blockquote><blockquote type="cite">...<br></blockquote><blockquote type="cite">I get the error:<br></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">Using default cluster template: medium Validating cluster template <br></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">settings...<br></blockquote></blockquote></blockquote></blockquote><blockquote type="cite">cluster.py:766 - ERROR - spot_bid must be integer or float<br></blockquote><blockquote type="cite">cli.py:243 - ERROR - settings for cluster template "medium" are not valid<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Is the way I've specified this wrong? Or perhaps, it's being loaded as a<br></blockquote><blockquote type="cite">string and isn't getting converted to a float internally?<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Cheers,<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Ben<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Benjamin Lambert<br></blockquote><blockquote type="cite">Ph.D. Student of Computer Science<br></blockquote><blockquote type="cite">Carnegie Mellon University<br></blockquote><blockquote type="cite"><a href="http://www.cs.cmu.edu/~belamber">www.cs.cmu.edu/~belamber</a><br></blockquote><blockquote type="cite">Mobile: 617-869-1844<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">_______________________________________________<br></blockquote><blockquote type="cite">StarCluster mailing list<br></blockquote><blockquote type="cite"><a href="mailto:StarCluster@mit.edu">StarCluster@mit.edu</a><br></blockquote><blockquote type="cite"><a href="http://mailman.mit.edu/mailman/listinfo/starcluster">http://mailman.mit.edu/mailman/listinfo/starcluster</a><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">_______________________________________________<br></blockquote><blockquote type="cite">StarCluster mailing list<br></blockquote><blockquote type="cite"><a href="mailto:StarCluster@mit.edu">StarCluster@mit.edu</a><br></blockquote><blockquote type="cite"><a href="http://mailman.mit.edu/mailman/listinfo/starcluster">http://mailman.mit.edu/mailman/listinfo/starcluster</a><br></blockquote><br></div></blockquote></div><br></div></div></body></html>