Hi Rayson, <div><br></div><div>Thanks again. Now even away from bidding, and using the same configuration file that connected successfully 3 days ago, </div><div><br></div><div>I am consistently now getting this error:</div>
<div><br></div><div><div>!!! ERROR - Unsupported: Your requested instance type (cg1.4xlarge) is not supported in your requested Availability Zone (us-east-1c). Please retry your request by not specifying an Availability Zone or choosing us-east-1a.</div>
<div><br></div><div><br></div><div>I do as it says, and remove any region specifications, or specify the "us-east-1a" region at says, and it still keep coming back with the same error, 3 days now without successful connection with bidding and without. I am not sure how to solve that</div>
<div><br></div><div>I appreciate your support very much. </div><div><br></div><div>manal</div><br><div class="gmail_quote">On 20 May 2012 12:48, Rayson Ho <span dir="ltr"><<a href="mailto:raysonlogin@gmail.com" target="_blank">raysonlogin@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">If you bid below the current spot price, then your request will take a<br>
long time (or never, if the spot price never goes lower than your bid<br>
price). The common practice is to bid above the current spot price and<br>
below the standard on-demand price to get the best chance of getting<br>
an instance and without over paying for spot instances.<br>
<br>
See the "Deciding on Your Spot Bidding Strategy" video tutorial<br>
created by AmazonWebServices:<br>
<br>
<a href="http://www.youtube.com/watch?v=WD9N73F3Fao" target="_blank">http://www.youtube.com/watch?v=WD9N73F3Fao</a><br>
<div class="im HOEnZb"><br>
Rayson<br>
<br>
================================<br>
Open Grid Scheduler / Grid Engine<br>
<a href="http://gridscheduler.sourceforge.net/" target="_blank">http://gridscheduler.sourceforge.net/</a><br>
<br>
Scalable Grid Engine Support Program<br>
<a href="http://www.scalablelogic.com/" target="_blank">http://www.scalablelogic.com/</a><br>
<br>
<br>
</div><div class="HOEnZb"><div class="h5">On Sat, May 19, 2012 at 10:30 PM, Manal Helal <<a href="mailto:manal.helal@gmail.com">manal.helal@gmail.com</a>> wrote:<br>
> Thanks Rayson, I just waited for too long on a one-time request, and<br>
> misunderstood the persistence feature, though it keeps on requesting during<br>
> waiting, not just after termination,<br>
><br>
> thanks again,<br>
><br>
> Manal<br>
><br>
><br>
> On 20 May 2012 10:29, Rayson Ho <<a href="mailto:raysonlogin@gmail.com">raysonlogin@gmail.com</a>> wrote:<br>
>><br>
>> Is there a real use case for persistent spot request?<br>
>><br>
>> <a href="http://aws.amazon.com/ec2/spot-instances/#1" target="_blank">http://aws.amazon.com/ec2/spot-instances/#1</a><br>
>><br>
>> "... with a persistent request, after your request has been satisfied<br>
>> and your instance has been terminated—by you or by Amazon EC2—your<br>
>> request will be submitted again automatically with the same parameters<br>
>> as your initial request. A persistent request will continue submitting<br>
>> the request until you cancel it..."<br>
>><br>
>> Rayson<br>
>><br>
>> ================================<br>
>> Open Grid Scheduler / Grid Engine<br>
>> <a href="http://gridscheduler.sourceforge.net/" target="_blank">http://gridscheduler.sourceforge.net/</a><br>
>><br>
>> Scalable Grid Engine Support Program<br>
>> <a href="http://www.scalablelogic.com/" target="_blank">http://www.scalablelogic.com/</a><br>
>><br>
>><br>
>> On Fri, May 18, 2012 at 10:03 PM, Manal Helal <<a href="mailto:manal.helal@gmail.com">manal.helal@gmail.com</a>><br>
>> wrote:<br>
>> > Hi,<br>
>> ><br>
>> > I also need information about the command line option for making a spot<br>
>> > request persistent. I am currently using:<br>
>> ><br>
>> > starcluster start -o -b mybid mycluster<br>
>> ><br>
>> > and in the AWS online GUI console, I see that this generates a one-time<br>
>> > spot<br>
>> > request not a persistent one. I googled on AWS or starcluster to find<br>
>> > out<br>
>> > how to change this after request or when I make the initial request,<br>
>> ><br>
>> > thank you very much for your support,<br>
>> ><br>
>> > Kind Regards,<br>
>> ><br>
>> > On 19 May 2012 11:17, Manal Helal <<a href="mailto:manalorama@gmail.com">manalorama@gmail.com</a>> wrote:<br>
>> >><br>
>> >> Hi Rayson,<br>
>> >><br>
>> >> Thank you very much for the extended explanations and for sharing your<br>
>> >> experience. I had to start fast, and read only quick-guides, and first<br>
>> >> two<br>
>> >> pages of everything, and googling what I need only,<br>
>> >><br>
>> >> thanks again,<br>
>> >><br>
>> >> Kind Regards,<br>
>> >><br>
>> >> Manal<br>
>> >><br>
>> >><br>
>> >> On 19 May 2012 02:47, Rayson Ho <<a href="mailto:raysonlogin@gmail.com">raysonlogin@gmail.com</a>> wrote:<br>
>> >>><br>
>> >>> Manal,<br>
>> >>><br>
>> >>> Here's my spending back in Feb. I am also in the free tier, BTW.<br>
>> >>><br>
>> >>> In that month, I used 27 hours of t1.micro & 1 hour of m1.small, thus<br>
>> >>> I got charged $ 0.09 ($0.085 rounded up!).<br>
>> >>><br>
>> >>> Amazon EC2 running Linux/UNIX<br>
>> >>> $0.00 per Micro Instance (t1.micro) instance-hour (or<br>
>> >>> partial<br>
>> >>> hour) under monthly free tier 27 Hrs 0.00<br>
>> >>> $0.085 per Small Instance (m1.small) instance-hour (or<br>
>> >>> partial<br>
>> >>> hour) 1 Hr 0.09<br>
>> >>><br>
>> >>> Amazon EC2 EBS<br>
>> >>> $0.00 per GB-month of provisioned storage under<br>
>> >>> monthly<br>
>> >>> free<br>
>> >>> tier 0.398 GB-Mo 0.00<br>
>> >>> $0.00 per 1 million I/O requests under monthly free<br>
>> >>> tier<br>
>> >>> 428,595 IOs 0.00<br>
>> >>><br>
>> >>> Elastic IP Addresses<br>
>> >>> $0.00 per Elastic IP address remap - first 100 remaps<br>
>> >>> /<br>
>> >>> month 1<br>
>> >>> Count 0.00<br>
>> >>><br>
>> >>><br>
>> >>> I am under the free EBS limit (it was then 10GB month, but the limit<br>
>> >>> got bumped up to 30 GB month recently). Note that you can use 60GB for<br>
>> >>> 15 days, then 0 GB for the rest of the month, and Amazon will not<br>
>> >>> charge you anything. I think Amazon has a very straightforward<br>
>> >>> pricing, and you don't need to worry about whether the EBS is used by<br>
>> >>> a t1.micro or a cg1.4xlarge or even just left there for backup...<br>
>> >>><br>
>> >>> Basically, feel free to mix and match free & non-free EC2 resource.<br>
>> >>><br>
>> >>> Rayson<br>
>> >>><br>
>> >>> ================================<br>
>> >>> Open Grid Scheduler / Grid Engine<br>
>> >>> <a href="http://gridscheduler.sourceforge.net/" target="_blank">http://gridscheduler.sourceforge.net/</a><br>
>> >>><br>
>> >>> Scalable Grid Engine Support Program<br>
>> >>> <a href="http://www.scalablelogic.com/" target="_blank">http://www.scalablelogic.com/</a><br>
>> >>><br>
>> >>><br>
>> >>><br>
>> >>> On Thu, May 17, 2012 at 9:59 PM, Rayson Ho <<a href="mailto:raysonlogin@gmail.com">raysonlogin@gmail.com</a>><br>
>> >>> wrote:<br>
>> >>> > So you are in the "free tier" indeed! Keep in mind that you get 30<br>
>> >>> > GB<br>
>> >>> > of free EBS storage, and if you are willing to pay, then you don't<br>
>> >>> > need to limit yourself to micro instances.<br>
>> >>> ><br>
>> >>> > For example, you can still use CG1 instances (which are not free<br>
>> >>> > even<br>
>> >>> > for those in the free tier) and you still get free 30GB of EBS<br>
>> >>> > storage. You basically can mix and match free & non-free stuff in<br>
>> >>> > AWS.<br>
>> >>> ><br>
>> >>> > Rayson<br>
>> >>> ><br>
>> >>> > ================================<br>
>> >>> > Open Grid Scheduler / Grid Engine<br>
>> >>> > <a href="http://gridscheduler.sourceforge.net/" target="_blank">http://gridscheduler.sourceforge.net/</a><br>
>> >>> ><br>
>> >>> > Scalable Grid Engine Support Program<br>
>> >>> > <a href="http://www.scalablelogic.com/" target="_blank">http://www.scalablelogic.com/</a><br>
>> >>> ><br>
>> >>> ><br>
>> >>> ><br>
>> >>> > On Thu, May 17, 2012 at 9:27 PM, Manal Helal <<a href="mailto:manalorama@gmail.com">manalorama@gmail.com</a>><br>
>> >>> > wrote:<br>
>> >>> >> Thank you very much Justin and Rayson for the explanation,<br>
>> >>> >><br>
>> >>> >> I actually started the cluster Yesterday and learned a few issues<br>
>> >>> >> by<br>
>> >>> >> try and<br>
>> >>> >> error, realising the already mounted EBS in the starcluster HVM<br>
>> >>> >> AMI,<br>
>> >>> >> but<br>
>> >>> >> terminates with it and needs downloading the data. I also realised<br>
>> >>> >> the<br>
>> >>> >> unfeasibility of the thought of downloading an image, updating it<br>
>> >>> >> on<br>
>> >>> >> my<br>
>> >>> >> machine, and then upload it after testing and everytime I need it.<br>
>> >>> >> Also<br>
>> >>> >> using a spotnode cost less, so I can experiment with less worries,<br>
>> >>> >> but<br>
>> >>> >> the<br>
>> >>> >> IO free-tier is small, and only tiny instances are in the free<br>
>> >>> >> tier,<br>
>> >>> >> and I<br>
>> >>> >> need cg1.4xlarge to use the GPU Cluster,<br>
>> >>> >><br>
>> >>> >> I tried the following steps:<br>
>> >>> >><br>
>> >>> >><br>
>> >>> >> <a href="http://instantbadger.blogspot.com.au/2009/09/how-to-create-and-save-ami-image-from.html" target="_blank">http://instantbadger.blogspot.com.au/2009/09/how-to-create-and-save-ami-image-from.html</a><br>
>> >>> >><br>
>> >>> >> to bundle, upload and register the customised running AMI after my<br>
>> >>> >> updates,<br>
>> >>> >> and the bundle took time indeed, but this seams like the<br>
>> >>> >> alternative<br>
>> >>> >> to the<br>
>> >>> >> scenario download/upload I was thinking of,<br>
>> >>> >><br>
>> >>> >> thanks again for your help,<br>
>> >>> >><br>
>> >>> >> Kind Regards,<br>
>> >>> >><br>
>> >>> >><br>
>> >>> >> On 18 May 2012 07:40, Rayson Ho <<a href="mailto:raysonlogin@gmail.com">raysonlogin@gmail.com</a>> wrote:<br>
>> >>> >>><br>
>> >>> >>> On Thu, May 17, 2012 at 5:23 PM, Justin Riley <<a href="mailto:jtriley@mit.edu">jtriley@mit.edu</a>><br>
>> >>> >>> wrote:<br>
>> >>> >>> > If you're trying to find a way to store data without having to<br>
>> >>> >>> > keep<br>
>> >>> >>> > the<br>
>> >>> >>> > cluster around there are much better approaches:<br>
>> >>> >>><br>
>> >>> >>> Manal,<br>
>> >>> >>><br>
>> >>> >>> Since you mentioned that you are new to AWS, you should be able to<br>
>> >>> >>> get<br>
>> >>> >>> the benefits of the free tier:<br>
>> >>> >>><br>
>> >>> >>> <a href="http://aws.amazon.com/free/" target="_blank">http://aws.amazon.com/free/</a><br>
>> >>> >>><br>
>> >>> >>> Basically, you can use 30 GB of EBS storage free of charge. On the<br>
>> >>> >>> other hand, if you pull data from AWS to your local harddrive<br>
>> >>> >>> everytime you are done with the instance, then you will need to<br>
>> >>> >>> pay<br>
>> >>> >>> for I/O & data transfer costs.<br>
>> >>> >>><br>
>> >>> >>><br>
>> >>> >>> Justin,<br>
>> >>> >>><br>
>> >>> >>> I am trying to find the version of the NVIDIA Driver & CUDA<br>
>> >>> >>> toolkit<br>
>> >>> >>> installed on the CG1 AMIs, but instead of booting up an instance<br>
>> >>> >>> to<br>
>> >>> >>> find out what's there, is there a page that lists the additional<br>
>> >>> >>> software packages installed in the AMIs?? So far, I could only<br>
>> >>> >>> find:<br>
>> >>> >>><br>
>> >>> >>> <a href="https://github.com/jtriley/StarCluster/issues/9" target="_blank">https://github.com/jtriley/StarCluster/issues/9</a><br>
>> >>> >>><br>
>> >>> >>> Rayson<br>
>> >>> >>><br>
>> >>> >>> ================================<br>
>> >>> >>> Open Grid Scheduler / Grid Engine<br>
>> >>> >>> <a href="http://gridscheduler.sourceforge.net/" target="_blank">http://gridscheduler.sourceforge.net/</a><br>
>> >>> >>><br>
>> >>> >>> Scalable Grid Engine Support Program<br>
>> >>> >>> <a href="http://www.scalablelogic.com/" target="_blank">http://www.scalablelogic.com/</a><br>
>> >>> >>><br>
>> >>> >>><br>
>> >>> >>><br>
>> >>> >>> ><br>
>> >>> >>> > 1. Use an external EBS volume with StarCluster and store your<br>
>> >>> >>> > data<br>
>> >>> >>> > there. This will allow you to terminate the cluster and still<br>
>> >>> >>> > preserve<br>
>> >>> >>> > your important data on the EBS volume. See:<br>
>> >>> >>> ><br>
>> >>> >>> > <a href="http://web.mit.edu/star/cluster/docs/latest/manual/volumes.html" target="_blank">http://web.mit.edu/star/cluster/docs/latest/manual/volumes.html</a><br>
>> >>> >>> ><br>
>> >>> >>> > 2. If you don't wish to use external EBS volumes at all you can<br>
>> >>> >>> > always<br>
>> >>> >>> > use starcluster's "get" command to manually download files from<br>
>> >>> >>> > the<br>
>> >>> >>> > cluster before terminating, e.g.:<br>
>> >>> >>> ><br>
>> >>> >>> > $ starcluster get mycluster /path/to/results.tar.gz<br>
>> >>> >>> ><br>
>> >>> >>> > Similarly you can upload files using the "put" command:<br>
>> >>> >>> ><br>
>> >>> >>> > $ starcluster put mycluster /path/to/local/file /remote/path/<br>
>> >>> >>> ><br>
>> >>> >>> > See the following doc for more details:<br>
>> >>> >>> ><br>
>> >>> >>> > <a href="http://web.mit.edu/star/cluster/docs/latest/manual/putget.html" target="_blank">http://web.mit.edu/star/cluster/docs/latest/manual/putget.html</a><br>
>> >>> >>> ><br>
>> >>> >>> >> 4. After knowing how to download the image, I will need the<br>
>> >>> >>> >> command<br>
>> >>> >>> >> to<br>
>> >>> >>> >> start a new cluster using the image I have stored on my<br>
>> >>> >>> >> local<br>
>> >>> >>> >> disk.<br>
>> >>> >>> >> I<br>
>> >>> >>> >> think I have seen something to install from an image to the<br>
>> >>> >>> >> cluster<br>
>> >>> >>> ><br>
>> >>> >>> > Launching a cluster using a locally stored disk image is not<br>
>> >>> >>> > supported.<br>
>> >>> >>> > The latency in doing this would be *insane*. Having to upload<br>
>> >>> >>> > multiple<br>
>> >>> >>> > GBs and register an AMI everytime would make starting a cluster<br>
>> >>> >>> > *extremely* time-consuming and error-prone.<br>
>> >>> >>> ><br>
>> >>> >>> > As mentioned above you're better off creating a new AMI for your<br>
>> >>> >>> > needs<br>
>> >>> >>> > and keeping the new AMI on Amazon. Then simply update your<br>
>> >>> >>> > StarCluster<br>
>> >>> >>> > config to use your new AMI. If you need to save data before<br>
>> >>> >>> > terminating<br>
>> >>> >>> > the cluster then either save your data to an external EBS volume<br>
>> >>> >>> > or<br>
>> >>> >>> > copy<br>
>> >>> >>> > the data using the 'get' command as mentined above.<br>
>> >>> >>> ><br>
>> >>> >>> > Hope that helps,<br>
>> >>> >>> ><br>
>> >>> >>> > ~Justin<br>
>> >>> >>> ><br>
>> >>> >>> > _______________________________________________<br>
>> >>> >>> > StarCluster mailing list<br>
>> >>> >>> > <a href="mailto:StarCluster@mit.edu">StarCluster@mit.edu</a><br>
>> >>> >>> > <a href="http://mailman.mit.edu/mailman/listinfo/starcluster" target="_blank">http://mailman.mit.edu/mailman/listinfo/starcluster</a><br>
>> >>> >>> ><br>
>> >>> >>><br>
>> >>> >>><br>
>> >>> >>><br>
>> >>> >>> --<br>
>> >>> >>> ==================================================<br>
>> >>> >>> Open Grid Scheduler - The Official Open Source Grid Engine<br>
>> >>> >>> <a href="http://gridscheduler.sourceforge.net/" target="_blank">http://gridscheduler.sourceforge.net/</a><br>
>> >>> >><br>
>> >>> >><br>
>> >>> >><br>
>> >>> >><br>
>> >>> >> --<br>
>> >>> >> Kind Regards,<br>
>> >>> >><br>
>> >>> >> Manal Helal<br>
>> >>> >><br>
>> >>> ><br>
>> >>> ><br>
>> >>> ><br>
>> >>> > --<br>
>> >>> > ==================================================<br>
>> >>> > Open Grid Scheduler - The Official Open Source Grid Engine<br>
>> >>> > <a href="http://gridscheduler.sourceforge.net/" target="_blank">http://gridscheduler.sourceforge.net/</a><br>
>> >>><br>
>> >>><br>
>> >>><br>
>> >>> --<br>
>> >>> ==================================================<br>
>> >>> Open Grid Scheduler - The Official Open Source Grid Engine<br>
>> >>> <a href="http://gridscheduler.sourceforge.net/" target="_blank">http://gridscheduler.sourceforge.net/</a><br>
>> >><br>
>> >><br>
>> >><br>
>> >><br>
>> ><br>
>> ><br>
>><br>
>><br>
>><br>
>> --<br>
>> ==================================================<br>
>> Open Grid Scheduler - The Official Open Source Grid Engine<br>
>> <a href="http://gridscheduler.sourceforge.net/" target="_blank">http://gridscheduler.sourceforge.net/</a><br>
><br>
><br>
><br>
><br>
<br>
<br>
<br>
--<br>
==================================================<br>
Open Grid Scheduler - The Official Open Source Grid Engine<br>
<a href="http://gridscheduler.sourceforge.net/" target="_blank">http://gridscheduler.sourceforge.net/</a><br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>
</div>