Thank you very much Justin and Rayson for the explanation, <div><br></div><div>I actually started the cluster Yesterday and learned a few issues by try and error, realising the already mounted EBS in the starcluster HVM AMI, but terminates with it and needs downloading the data. I also realised the unfeasibility of the thought of downloading an image, updating it on my machine, and then upload it after testing and everytime I need it. Also using a spotnode cost less, so I can experiment with less worries, but the IO free-tier is small, and only tiny instances are in the free tier, and I need cg1.4xlarge to use the GPU Cluster, </div>
<div><br></div><div>I tried the following steps:</div><div><a href="http://instantbadger.blogspot.com.au/2009/09/how-to-create-and-save-ami-image-from.html">http://instantbadger.blogspot.com.au/2009/09/how-to-create-and-save-ami-image-from.html</a><br>
<br></div><div>to bundle, upload and register the customised running AMI after my updates, and the bundle took time indeed, but this seams like the alternative to the scenario download/upload I was thinking of, </div><div>
<br></div><div>thanks again for your help, </div><div><br></div><div>Kind Regards, </div><div><br></div><div><br><div class="gmail_quote">On 18 May 2012 07:40, Rayson Ho <span dir="ltr"><<a href="mailto:raysonlogin@gmail.com" target="_blank">raysonlogin@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">On Thu, May 17, 2012 at 5:23 PM, Justin Riley <<a href="mailto:jtriley@mit.edu">jtriley@mit.edu</a>> wrote:<br>
> If you're trying to find a way to store data without having to keep the<br>
> cluster around there are much better approaches:<br>
<br>
</div>Manal,<br>
<br>
Since you mentioned that you are new to AWS, you should be able to get<br>
the benefits of the free tier:<br>
<br>
<a href="http://aws.amazon.com/free/" target="_blank">http://aws.amazon.com/free/</a><br>
<br>
Basically, you can use 30 GB of EBS storage free of charge. On the<br>
other hand, if you pull data from AWS to your local harddrive<br>
everytime you are done with the instance, then you will need to pay<br>
for I/O & data transfer costs.<br>
<br>
<br>
Justin,<br>
<br>
I am trying to find the version of the NVIDIA Driver & CUDA toolkit<br>
installed on the CG1 AMIs, but instead of booting up an instance to<br>
find out what's there, is there a page that lists the additional<br>
software packages installed in the AMIs?? So far, I could only find:<br>
<br>
<a href="https://github.com/jtriley/StarCluster/issues/9" target="_blank">https://github.com/jtriley/StarCluster/issues/9</a><br>
<br>
Rayson<br>
<br>
================================<br>
Open Grid Scheduler / Grid Engine<br>
<a href="http://gridscheduler.sourceforge.net/" target="_blank">http://gridscheduler.sourceforge.net/</a><br>
<br>
Scalable Grid Engine Support Program<br>
<a href="http://www.scalablelogic.com/" target="_blank">http://www.scalablelogic.com/</a><br>
<div><div class="h5"><br>
<br>
<br>
><br>
> 1. Use an external EBS volume with StarCluster and store your data<br>
> there. This will allow you to terminate the cluster and still preserve<br>
> your important data on the EBS volume. See:<br>
><br>
> <a href="http://web.mit.edu/star/cluster/docs/latest/manual/volumes.html" target="_blank">http://web.mit.edu/star/cluster/docs/latest/manual/volumes.html</a><br>
><br>
> 2. If you don't wish to use external EBS volumes at all you can always<br>
> use starcluster's "get" command to manually download files from the<br>
> cluster before terminating, e.g.:<br>
><br>
> $ starcluster get mycluster /path/to/results.tar.gz<br>
><br>
> Similarly you can upload files using the "put" command:<br>
><br>
> $ starcluster put mycluster /path/to/local/file /remote/path/<br>
><br>
> See the following doc for more details:<br>
><br>
> <a href="http://web.mit.edu/star/cluster/docs/latest/manual/putget.html" target="_blank">http://web.mit.edu/star/cluster/docs/latest/manual/putget.html</a><br>
><br>
>> 4. After knowing how to download the image, I will need the command to<br>
>> start a new cluster using the image I have stored on my local disk. I<br>
>> think I have seen something to install from an image to the cluster<br>
><br>
> Launching a cluster using a locally stored disk image is not supported.<br>
> The latency in doing this would be *insane*. Having to upload multiple<br>
> GBs and register an AMI everytime would make starting a cluster<br>
> *extremely* time-consuming and error-prone.<br>
><br>
> As mentioned above you're better off creating a new AMI for your needs<br>
> and keeping the new AMI on Amazon. Then simply update your StarCluster<br>
> config to use your new AMI. If you need to save data before terminating<br>
> the cluster then either save your data to an external EBS volume or copy<br>
> the data using the 'get' command as mentined above.<br>
><br>
> Hope that helps,<br>
><br>
> ~Justin<br>
><br>
</div></div>> _______________________________________________<br>
> StarCluster mailing list<br>
> <a href="mailto:StarCluster@mit.edu">StarCluster@mit.edu</a><br>
> <a href="http://mailman.mit.edu/mailman/listinfo/starcluster" target="_blank">http://mailman.mit.edu/mailman/listinfo/starcluster</a><br>
><br>
<span class="HOEnZb"><font color="#888888"><br>
<br>
<br>
--<br>
==================================================<br>
Open Grid Scheduler - The Official Open Source Grid Engine<br>
<a href="http://gridscheduler.sourceforge.net/" target="_blank">http://gridscheduler.sourceforge.net/</a><br>
</font></span></blockquote></div><br><br clear="all"><div><br></div>-- <br><div dir="ltr">Kind Regards, </div><div dir="ltr"><br></div><div dir="ltr">Manal Helal</div><br>
</div>