[StarCluster] newbie problems

[Rayson Ho]

So you are in the "free tier" indeed! Keep in mind that you get 30 GB
of free EBS storage, and if you are willing to pay, then you don't
need to limit yourself to micro instances.

For example, you can still use CG1 instances (which are not free even
for those in the free tier) and you still get free 30GB of EBS
storage. You basically can mix and match free & non-free stuff in AWS.

Rayson

================================
Open Grid Scheduler / Grid Engine
http://gridscheduler.sourceforge.net/

Scalable Grid Engine Support Program
http://www.scalablelogic.com/



On Thu, May 17, 2012 at 9:27 PM, Manal Helal <manalorama at gmail.com> wrote:
> Thank you very much Justin and Rayson for the explanation,
>
> I actually started the cluster Yesterday and learned a few issues by try and
> error, realising the already mounted EBS in the starcluster HVM AMI, but
> terminates with it and needs downloading the data. I also realised the
> unfeasibility of the thought of downloading an image, updating it on my
> machine, and then upload it after testing and everytime I need it. Also
> using a spotnode cost less, so I can experiment with less worries, but the
> IO free-tier is small, and only tiny instances are in the free tier, and I
> need cg1.4xlarge to use the GPU Cluster,
>
> I tried the following steps:
> http://instantbadger.blogspot.com.au/2009/09/how-to-create-and-save-ami-image-from.html
>
> to bundle, upload and register the customised running AMI after my updates,
> and the bundle took time indeed, but this seams like the alternative to the
> scenario download/upload I was thinking of,
>
> thanks again for your help,
>
> Kind Regards,
>
>
> On 18 May 2012 07:40, Rayson Ho <raysonlogin at gmail.com> wrote:
>>
>> On Thu, May 17, 2012 at 5:23 PM, Justin Riley <jtriley at mit.edu> wrote:
>> > If you're trying to find a way to store data without having to keep the
>> > cluster around there are much better approaches:
>>
>> Manal,
>>
>> Since you mentioned that you are new to AWS, you should be able to get
>> the benefits of the free tier:
>>
>> http://aws.amazon.com/free/
>>
>> Basically, you can use 30 GB of EBS storage free of charge. On the
>> other hand, if you pull data from AWS to your local harddrive
>> everytime you are done with the instance, then you will need to pay
>> for I/O & data transfer costs.
>>
>>
>> Justin,
>>
>> I am trying to find the version of the NVIDIA Driver & CUDA toolkit
>> installed on the CG1 AMIs, but instead of booting up an instance to
>> find out what's there, is there a page that lists the additional
>> software packages installed in the AMIs?? So far, I could only find:
>>
>> https://github.com/jtriley/StarCluster/issues/9
>>
>> Rayson
>>
>> ================================
>> Open Grid Scheduler / Grid Engine
>> http://gridscheduler.sourceforge.net/
>>
>> Scalable Grid Engine Support Program
>> http://www.scalablelogic.com/
>>
>>
>>
>> >
>> > 1. Use an external EBS volume with StarCluster and store your data
>> > there. This will allow you to terminate the cluster and still preserve
>> > your important data on the EBS volume. See:
>> >
>> > http://web.mit.edu/star/cluster/docs/latest/manual/volumes.html
>> >
>> > 2. If you don't wish to use external EBS volumes at all you can always
>> > use starcluster's "get" command to manually download files from the
>> > cluster before terminating, e.g.:
>> >
>> > $ starcluster get mycluster /path/to/results.tar.gz
>> >
>> > Similarly you can upload files using the "put" command:
>> >
>> > $ starcluster put mycluster /path/to/local/file /remote/path/
>> >
>> > See the following doc for more details:
>> >
>> > http://web.mit.edu/star/cluster/docs/latest/manual/putget.html
>> >
>> >>    4. After knowing how to download the image, I will need the command
>> >> to
>> >>    start a new cluster using the image I have stored on my local disk.
>> >> I
>> >>    think I have seen something to install from an image to the cluster
>> >
>> > Launching a cluster using a locally stored disk image is not supported.
>> > The latency in doing this would be *insane*. Having to upload multiple
>> > GBs and register an AMI everytime would make starting a cluster
>> > *extremely* time-consuming and error-prone.
>> >
>> > As mentioned above you're better off creating a new AMI for your needs
>> > and keeping the new AMI on Amazon. Then simply update your StarCluster
>> > config to use your new AMI. If you need to save data before terminating
>> > the cluster then either save your data to an external EBS volume or copy
>> > the data using the 'get' command as mentined above.
>> >
>> > Hope that helps,
>> >
>> > ~Justin
>> >
>> > _______________________________________________
>> > StarCluster mailing list
>> > StarCluster at mit.edu
>> > http://mailman.mit.edu/mailman/listinfo/starcluster
>> >
>>
>>
>>
>> --
>> ==================================================
>> Open Grid Scheduler - The Official Open Source Grid Engine
>> http://gridscheduler.sourceforge.net/
>
>
>
>
> --
> Kind Regards,
>
> Manal Helal
>



-- 
==================================================
Open Grid Scheduler - The Official Open Source Grid Engine
http://gridscheduler.sourceforge.net/