[StarCluster] Setting a spot bid
Benjamin Lambert
benlambert at cmu.edu
Tue Apr 19 01:43:01 EDT 2011
Thanks Justin. When I asked this question, I had *thought* I was already running the development version, but the Python/Mac setup had thoroughly confused me. Several weeks ago I realized this and upgraded to the GIT version.
That has been working very well for me, including the SPOT_BID setting. I've tried the load balancing feature as well, and 3 of the 4 times I tried it, it worked perfectly (not sure what happened the other time). I ended up hacking the loadbalancer code to more aggressively add nodes. I made a few other changes; they're probably not worth contributing, but I'd be glad to if you like. I'll paste one below that might be helpful though, for the functions: "get_running_jobs" "get_queued_jobs" (Lemme know the best way to send a patch if you want).
Having used it a bunch, the only other feedback I have is that it would be nice if "addnode" took a parameter for *how many* nodes to add, so you could add 10 in one go. E.g. starcluster addnode -n 10
The doc update and release will be very nice, because the system is already much more capable than the website would make it seem. I have been recommending it to others. Starcluster rocks!
Ben
In: StarCluster/starcluster/balancers/sge/__init__.py
I think this requires a change to parse_qstat() as well. The idea is that a job may be "pending", but if it's "state" is "hqw", then it's pending, but it's also waiting for something else to finish (or it's been put on hold manually). Presumably, you don't want to launch new instances for pending jobs that are on hold anyway.
def get_running_jobs(self):
"returns an array of the running jobs, values stored in dictionary"
running = []
for j in self.jobs:
#if j['job_state'] == u'running':
if j['state'] == u'r':
running.append(j)
return running
def get_queued_jobs(self):
"returns an array of the queued jobs, values stored in dictionary"
queued = []
for j in self.jobs:
#if j['job_state'] == u'pending':
if j['state'] == u'qw':
queued.append(j)
return queued
On Apr 19, 2011, at 12:51 AM, Justin Riley wrote:
> Hi Ben/Simms,
>
> Unfortunately the spot bid is not a supported config option in the current stable version of StarCluster (0.91.2). I've added the ability to specify a SPOT_BID setting within a cluster template in the latest development code if you're interested in testing. This functionality will be available in the 0.92 release. It's taking me forever to release this next version due to other projects at work but I intend to release soon. My apologies to everyone for the extreme delay. At this point I need to update the docs and then it's ready. I plan to have a *much* more frequent incremental release cycle after this next big release comes out.
>
> ~Justin
>
>
> On Mar 21, 2011, at 5:49 PM, Justin Simms justin.simms at gmail.com wrote:
>
>> Ben,
>>
>> As far as I can tell, it seems to be the case that there is no conversion
>> from the string type to the float or int type that is expected. The way that
>> you have provided your argument is correct.
>>
>> jts
>>
>> -----Original Message-----
>> From: starcluster-bounces at mit.edu [mailto:starcluster-bounces at mit.edu] On
>> Behalf Of Benjamin Lambert
>> Sent: Monday, March 21, 2011 3:30 PM
>> To: starcluster at mit.edu
>> Subject: [StarCluster] Setting a spot bid
>>
>> Hi StarCluster folks,
>>
>> I just came across StarCluster last week. We've been having some trouble
>> with a local physical SGE cluster, and I needed a backup/replacement. I
>> made the mistake of trying to set up an existing Ubuntu AMI to work in
>> StarCluster, without knowing what I was getting in to. But once I gave up
>> on that and started fresh from the default SG Ubuntu 10.04 64-bit image, I
>> got things up and running smoothly. Very nice work!
>>
>> I'm looking forward to the features planned for version 0.92, especially
>> elastic load balancing, add/remove nodes, and EBS-backed AMIs. Any estimate
>> when we might see 0.92?
>>
>> I am having a slight issue with bidding for spot instances. It works just
>> fine if I request it through the command line:
>> starcluster start --bid=0.50 cluster
>>
>> But, if I put the bid in my .starcluster/config file like so:
>> [cluster medium]
>> KEYNAME = ...
>> SPOT_BID = 0.35
>> ...
>> I get the error:
>>>>> Using default cluster template: medium Validating cluster template
>>>>> settings...
>> cluster.py:766 - ERROR - spot_bid must be integer or float
>> cli.py:243 - ERROR - settings for cluster template "medium" are not valid
>>
>> Is the way I've specified this wrong? Or perhaps, it's being loaded as a
>> string and isn't getting converted to a float internally?
>>
>> Cheers,
>>
>> Ben
>>
>>
>> Benjamin Lambert
>> Ph.D. Student of Computer Science
>> Carnegie Mellon University
>> www.cs.cmu.edu/~belamber
>> Mobile: 617-869-1844
>>
>>
>> _______________________________________________
>> StarCluster mailing list
>> StarCluster at mit.edu
>> http://mailman.mit.edu/mailman/listinfo/starcluster
>>
>> _______________________________________________
>> StarCluster mailing list
>> StarCluster at mit.edu
>> http://mailman.mit.edu/mailman/listinfo/starcluster
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/pipermail/starcluster/attachments/20110419/d847025f/attachment.htm
More information about the StarCluster
mailing list