[Starcluster] EC2 AMI creation

Mark J. Pearrow mjp at MIT.EDU
Tue Dec 8 12:46:38 EST 2009


Hi Justin,

Thanks for the reply. I had installed StarCluster using easy_install, but didn't find anything called create_image.py. So I downloaded the latest StarCluster from this link:

http://web.mit.edu/stardev/cluster/starcluster-0.90.tar.gz

I then unpacked it - but there is no utils directory. Is there some other package I should download? Also, do I need to run ElasticFox on the same computer where I am running create_image.py? 

mjp


On Dec 8, 2009, at 11:55 AM, Justin Riley wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hi Mark,
> 
> I've cc'd the StarCluster mailing list on this email, hope you don't mind.
> 
> First off, congrats on receiving AWS funding for your research and I'm
> delighted you're choosing to use StarCluster to jump start your work on
> EC2 :D
> 
> To answer your question concerning how to rebundle, if you download and
> unpack StarCluster you will see a folder called utils. Inside of that
> folder is a script called create_image.py.  Running:
> 
> python create_image.py --help
> 
> Should give you enough options to be self-explanatory. After it
> completes you will have a new AMI available in ElasticFox (you ARE using
> ElasticFox right? ;). To get StarCluster to use your new AMI, simply put
> this new AMI id in your ~/.starclustercfg config file for
> MASTER_IMAGE_ID/NODE_IMAGE_ID. Let me know if you need any help with
> this command, I'd be glad to help.
> 
> Concerning creating persistent EBS, it's very easy to have it mounted on
> /home on the entire cluster (via nfs) after the EBS volume has been
> created. Right now the process for creating a new EBS for StarCluster is
> a bit cumbersome but well-documented here:
> 
> http://web.mit.edu/stardev/cluster/documentation/ebsinit.html
> 
> I'm currently working on a new version of StarCluster that will make it
> considerably easier to create both new AMIs as well as new EBS volumes
> (which concerns both of your questions above). Stay Tuned!
> 
> Concering external LDAP authentication support on an AMI, I've not
> looked into this but it certainly could be possible. If you find
> something let me know and I'll look into it and consider adding it in as
> optional functionality in StarCluster.
> 
> BTW, I'm doing a session over MIT's IAP period on using StarCluster if
> you or anyone you know would be interested in attending. Details are
> below. We should be in the IAP calendar within a few days or so.
> 
> Using StarCluster:  A Toolbox for Managing Amazon Elastic Compute Clouds
> 
> DATES:  Tuesday, January 19, 2010
>        Tuesday, January 26, 2010
> 
> TIME:   11:00 a.m. – 4:00 p.m.
> 
> ROOM:   14-0637 (17 ATHENA Workstations)
> 
> Thanks for your interest in StarCluster!
> 
> ~Justin
> 
> Mark J. Pearrow wrote:
>> Hi Justin,
>> 
>> I was at the StarCluster presentation at the Stata Center back in September, and back then started experimenting with starcluster. I also applied for a research grant, which I just received, so I'm planning to start moving ahead with use. I have a question that is perhaps less starcluster-related: how do I modify the existing AMI and save it? And then how would I make star cluster launch that instance going forward?
>> 
>> I'm planning to create a persistent elastic block store and keep a lot of the neuroimaging software we use there, and hopefully I can mount that volume on all my instances.
>> 
>> Do you happen to know if it's possible to use external authentication methods on an AMI? i.e. can I use my LDAP server that authenticates my local cluster here?
>> 
>> Thanks,
>> 
>> mjp
>> 
> 
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.11 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
> 
> iEYEARECAAYFAksehPQACgkQ4llAkMfDcrm8ewCeODwBEvzzS7m1BeKCdRK+Cnm0
> FI8AniFz5cRSzov7Oj+/RKYDf9fL9ws2
> =b4L4
> -----END PGP SIGNATURE-----





More information about the StarCluster mailing list