Thanks Rayson, I just waited for too long on a one-time request, and misunderstood the persistence feature, though it keeps on requesting during waiting, not just after termination, <div><br></div><div>thanks again, </div>
<div><br></div><div>Manal<br><br><div class="gmail_quote">On 20 May 2012 10:29, Rayson Ho <span dir="ltr"><<a href="mailto:raysonlogin@gmail.com" target="_blank">raysonlogin@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Is there a real use case for persistent spot request?<br>
<br>
<a href="http://aws.amazon.com/ec2/spot-instances/#1" target="_blank">http://aws.amazon.com/ec2/spot-instances/#1</a><br>
<br>
"... with a persistent request, after your request has been satisfied<br>
and your instance has been terminated—by you or by Amazon EC2—your<br>
request will be submitted again automatically with the same parameters<br>
as your initial request. A persistent request will continue submitting<br>
the request until you cancel it..."<br>
<div class="im HOEnZb"><br>
Rayson<br>
<br>
================================<br>
Open Grid Scheduler / Grid Engine<br>
<a href="http://gridscheduler.sourceforge.net/" target="_blank">http://gridscheduler.sourceforge.net/</a><br>
<br>
Scalable Grid Engine Support Program<br>
<a href="http://www.scalablelogic.com/" target="_blank">http://www.scalablelogic.com/</a><br>
<br>
<br>
</div><div class="HOEnZb"><div class="h5">On Fri, May 18, 2012 at 10:03 PM, Manal Helal <<a href="mailto:manal.helal@gmail.com">manal.helal@gmail.com</a>> wrote:<br>
> Hi,<br>
><br>
> I also need information about the command line option for making a spot<br>
> request persistent. I am currently using:<br>
><br>
> starcluster start -o -b mybid mycluster<br>
><br>
> and in the AWS online GUI console, I see that this generates a one-time spot<br>
> request not a persistent one. I googled on AWS or starcluster to find out<br>
> how to change this after request or when I make the initial request,<br>
><br>
> thank you very much for your support,<br>
><br>
> Kind Regards,<br>
><br>
> On 19 May 2012 11:17, Manal Helal <<a href="mailto:manalorama@gmail.com">manalorama@gmail.com</a>> wrote:<br>
>><br>
>> Hi Rayson,<br>
>><br>
>> Thank you very much for the extended explanations and for sharing your<br>
>> experience. I had to start fast, and read only quick-guides, and first two<br>
>> pages of everything, and googling what I need only,<br>
>><br>
>> thanks again,<br>
>><br>
>> Kind Regards,<br>
>><br>
>> Manal<br>
>><br>
>><br>
>> On 19 May 2012 02:47, Rayson Ho <<a href="mailto:raysonlogin@gmail.com">raysonlogin@gmail.com</a>> wrote:<br>
>>><br>
>>> Manal,<br>
>>><br>
>>> Here's my spending back in Feb. I am also in the free tier, BTW.<br>
>>><br>
>>> In that month, I used 27 hours of t1.micro & 1 hour of m1.small, thus<br>
>>> I got charged $ 0.09 ($0.085 rounded up!).<br>
>>><br>
>>> Amazon EC2 running Linux/UNIX<br>
>>> $0.00 per Micro Instance (t1.micro) instance-hour (or<br>
>>> partial<br>
>>> hour) under monthly free tier 27 Hrs 0.00<br>
>>> $0.085 per Small Instance (m1.small) instance-hour (or<br>
>>> partial<br>
>>> hour) 1 Hr 0.09<br>
>>><br>
>>> Amazon EC2 EBS<br>
>>> $0.00 per GB-month of provisioned storage under monthly<br>
>>> free<br>
>>> tier 0.398 GB-Mo 0.00<br>
>>> $0.00 per 1 million I/O requests under monthly free tier<br>
>>> 428,595 IOs 0.00<br>
>>><br>
>>> Elastic IP Addresses<br>
>>> $0.00 per Elastic IP address remap - first 100 remaps /<br>
>>> month 1<br>
>>> Count 0.00<br>
>>><br>
>>><br>
>>> I am under the free EBS limit (it was then 10GB month, but the limit<br>
>>> got bumped up to 30 GB month recently). Note that you can use 60GB for<br>
>>> 15 days, then 0 GB for the rest of the month, and Amazon will not<br>
>>> charge you anything. I think Amazon has a very straightforward<br>
>>> pricing, and you don't need to worry about whether the EBS is used by<br>
>>> a t1.micro or a cg1.4xlarge or even just left there for backup...<br>
>>><br>
>>> Basically, feel free to mix and match free & non-free EC2 resource.<br>
>>><br>
>>> Rayson<br>
>>><br>
>>> ================================<br>
>>> Open Grid Scheduler / Grid Engine<br>
>>> <a href="http://gridscheduler.sourceforge.net/" target="_blank">http://gridscheduler.sourceforge.net/</a><br>
>>><br>
>>> Scalable Grid Engine Support Program<br>
>>> <a href="http://www.scalablelogic.com/" target="_blank">http://www.scalablelogic.com/</a><br>
>>><br>
>>><br>
>>><br>
>>> On Thu, May 17, 2012 at 9:59 PM, Rayson Ho <<a href="mailto:raysonlogin@gmail.com">raysonlogin@gmail.com</a>> wrote:<br>
>>> > So you are in the "free tier" indeed! Keep in mind that you get 30 GB<br>
>>> > of free EBS storage, and if you are willing to pay, then you don't<br>
>>> > need to limit yourself to micro instances.<br>
>>> ><br>
>>> > For example, you can still use CG1 instances (which are not free even<br>
>>> > for those in the free tier) and you still get free 30GB of EBS<br>
>>> > storage. You basically can mix and match free & non-free stuff in AWS.<br>
>>> ><br>
>>> > Rayson<br>
>>> ><br>
>>> > ================================<br>
>>> > Open Grid Scheduler / Grid Engine<br>
>>> > <a href="http://gridscheduler.sourceforge.net/" target="_blank">http://gridscheduler.sourceforge.net/</a><br>
>>> ><br>
>>> > Scalable Grid Engine Support Program<br>
>>> > <a href="http://www.scalablelogic.com/" target="_blank">http://www.scalablelogic.com/</a><br>
>>> ><br>
>>> ><br>
>>> ><br>
>>> > On Thu, May 17, 2012 at 9:27 PM, Manal Helal <<a href="mailto:manalorama@gmail.com">manalorama@gmail.com</a>><br>
>>> > wrote:<br>
>>> >> Thank you very much Justin and Rayson for the explanation,<br>
>>> >><br>
>>> >> I actually started the cluster Yesterday and learned a few issues by<br>
>>> >> try and<br>
>>> >> error, realising the already mounted EBS in the starcluster HVM AMI,<br>
>>> >> but<br>
>>> >> terminates with it and needs downloading the data. I also realised the<br>
>>> >> unfeasibility of the thought of downloading an image, updating it on<br>
>>> >> my<br>
>>> >> machine, and then upload it after testing and everytime I need it.<br>
>>> >> Also<br>
>>> >> using a spotnode cost less, so I can experiment with less worries, but<br>
>>> >> the<br>
>>> >> IO free-tier is small, and only tiny instances are in the free tier,<br>
>>> >> and I<br>
>>> >> need cg1.4xlarge to use the GPU Cluster,<br>
>>> >><br>
>>> >> I tried the following steps:<br>
>>> >><br>
>>> >> <a href="http://instantbadger.blogspot.com.au/2009/09/how-to-create-and-save-ami-image-from.html" target="_blank">http://instantbadger.blogspot.com.au/2009/09/how-to-create-and-save-ami-image-from.html</a><br>
>>> >><br>
>>> >> to bundle, upload and register the customised running AMI after my<br>
>>> >> updates,<br>
>>> >> and the bundle took time indeed, but this seams like the alternative<br>
>>> >> to the<br>
>>> >> scenario download/upload I was thinking of,<br>
>>> >><br>
>>> >> thanks again for your help,<br>
>>> >><br>
>>> >> Kind Regards,<br>
>>> >><br>
>>> >><br>
>>> >> On 18 May 2012 07:40, Rayson Ho <<a href="mailto:raysonlogin@gmail.com">raysonlogin@gmail.com</a>> wrote:<br>
>>> >>><br>
>>> >>> On Thu, May 17, 2012 at 5:23 PM, Justin Riley <<a href="mailto:jtriley@mit.edu">jtriley@mit.edu</a>><br>
>>> >>> wrote:<br>
>>> >>> > If you're trying to find a way to store data without having to keep<br>
>>> >>> > the<br>
>>> >>> > cluster around there are much better approaches:<br>
>>> >>><br>
>>> >>> Manal,<br>
>>> >>><br>
>>> >>> Since you mentioned that you are new to AWS, you should be able to<br>
>>> >>> get<br>
>>> >>> the benefits of the free tier:<br>
>>> >>><br>
>>> >>> <a href="http://aws.amazon.com/free/" target="_blank">http://aws.amazon.com/free/</a><br>
>>> >>><br>
>>> >>> Basically, you can use 30 GB of EBS storage free of charge. On the<br>
>>> >>> other hand, if you pull data from AWS to your local harddrive<br>
>>> >>> everytime you are done with the instance, then you will need to pay<br>
>>> >>> for I/O & data transfer costs.<br>
>>> >>><br>
>>> >>><br>
>>> >>> Justin,<br>
>>> >>><br>
>>> >>> I am trying to find the version of the NVIDIA Driver & CUDA toolkit<br>
>>> >>> installed on the CG1 AMIs, but instead of booting up an instance to<br>
>>> >>> find out what's there, is there a page that lists the additional<br>
>>> >>> software packages installed in the AMIs?? So far, I could only find:<br>
>>> >>><br>
>>> >>> <a href="https://github.com/jtriley/StarCluster/issues/9" target="_blank">https://github.com/jtriley/StarCluster/issues/9</a><br>
>>> >>><br>
>>> >>> Rayson<br>
>>> >>><br>
>>> >>> ================================<br>
>>> >>> Open Grid Scheduler / Grid Engine<br>
>>> >>> <a href="http://gridscheduler.sourceforge.net/" target="_blank">http://gridscheduler.sourceforge.net/</a><br>
>>> >>><br>
>>> >>> Scalable Grid Engine Support Program<br>
>>> >>> <a href="http://www.scalablelogic.com/" target="_blank">http://www.scalablelogic.com/</a><br>
>>> >>><br>
>>> >>><br>
>>> >>><br>
>>> >>> ><br>
>>> >>> > 1. Use an external EBS volume with StarCluster and store your data<br>
>>> >>> > there. This will allow you to terminate the cluster and still<br>
>>> >>> > preserve<br>
>>> >>> > your important data on the EBS volume. See:<br>
>>> >>> ><br>
>>> >>> > <a href="http://web.mit.edu/star/cluster/docs/latest/manual/volumes.html" target="_blank">http://web.mit.edu/star/cluster/docs/latest/manual/volumes.html</a><br>
>>> >>> ><br>
>>> >>> > 2. If you don't wish to use external EBS volumes at all you can<br>
>>> >>> > always<br>
>>> >>> > use starcluster's "get" command to manually download files from the<br>
>>> >>> > cluster before terminating, e.g.:<br>
>>> >>> ><br>
>>> >>> > $ starcluster get mycluster /path/to/results.tar.gz<br>
>>> >>> ><br>
>>> >>> > Similarly you can upload files using the "put" command:<br>
>>> >>> ><br>
>>> >>> > $ starcluster put mycluster /path/to/local/file /remote/path/<br>
>>> >>> ><br>
>>> >>> > See the following doc for more details:<br>
>>> >>> ><br>
>>> >>> > <a href="http://web.mit.edu/star/cluster/docs/latest/manual/putget.html" target="_blank">http://web.mit.edu/star/cluster/docs/latest/manual/putget.html</a><br>
>>> >>> ><br>
>>> >>> >> 4. After knowing how to download the image, I will need the<br>
>>> >>> >> command<br>
>>> >>> >> to<br>
>>> >>> >> start a new cluster using the image I have stored on my local<br>
>>> >>> >> disk.<br>
>>> >>> >> I<br>
>>> >>> >> think I have seen something to install from an image to the<br>
>>> >>> >> cluster<br>
>>> >>> ><br>
>>> >>> > Launching a cluster using a locally stored disk image is not<br>
>>> >>> > supported.<br>
>>> >>> > The latency in doing this would be *insane*. Having to upload<br>
>>> >>> > multiple<br>
>>> >>> > GBs and register an AMI everytime would make starting a cluster<br>
>>> >>> > *extremely* time-consuming and error-prone.<br>
>>> >>> ><br>
>>> >>> > As mentioned above you're better off creating a new AMI for your<br>
>>> >>> > needs<br>
>>> >>> > and keeping the new AMI on Amazon. Then simply update your<br>
>>> >>> > StarCluster<br>
>>> >>> > config to use your new AMI. If you need to save data before<br>
>>> >>> > terminating<br>
>>> >>> > the cluster then either save your data to an external EBS volume or<br>
>>> >>> > copy<br>
>>> >>> > the data using the 'get' command as mentined above.<br>
>>> >>> ><br>
>>> >>> > Hope that helps,<br>
>>> >>> ><br>
>>> >>> > ~Justin<br>
>>> >>> ><br>
>>> >>> > _______________________________________________<br>
>>> >>> > StarCluster mailing list<br>
>>> >>> > <a href="mailto:StarCluster@mit.edu">StarCluster@mit.edu</a><br>
>>> >>> > <a href="http://mailman.mit.edu/mailman/listinfo/starcluster" target="_blank">http://mailman.mit.edu/mailman/listinfo/starcluster</a><br>
>>> >>> ><br>
>>> >>><br>
>>> >>><br>
>>> >>><br>
>>> >>> --<br>
>>> >>> ==================================================<br>
>>> >>> Open Grid Scheduler - The Official Open Source Grid Engine<br>
>>> >>> <a href="http://gridscheduler.sourceforge.net/" target="_blank">http://gridscheduler.sourceforge.net/</a><br>
>>> >><br>
>>> >><br>
>>> >><br>
>>> >><br>
>>> >> --<br>
>>> >> Kind Regards,<br>
>>> >><br>
>>> >> Manal Helal<br>
>>> >><br>
>>> ><br>
>>> ><br>
>>> ><br>
>>> > --<br>
>>> > ==================================================<br>
>>> > Open Grid Scheduler - The Official Open Source Grid Engine<br>
>>> > <a href="http://gridscheduler.sourceforge.net/" target="_blank">http://gridscheduler.sourceforge.net/</a><br>
>>><br>
>>><br>
>>><br>
>>> --<br>
>>> ==================================================<br>
>>> Open Grid Scheduler - The Official Open Source Grid Engine<br>
>>> <a href="http://gridscheduler.sourceforge.net/" target="_blank">http://gridscheduler.sourceforge.net/</a><br>
>><br>
>><br>
>><br>
>><br>
><br>
><br>
<br>
<br>
<br>
--<br>
==================================================<br>
Open Grid Scheduler - The Official Open Source Grid Engine<br>
<a href="http://gridscheduler.sourceforge.net/" target="_blank">http://gridscheduler.sourceforge.net/</a><br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>
</div>